[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:17 dist-test-slave-fzjm krb5kdc[1160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:17 dist-test-slave-fzjm krb5kdc[1160](info): set up 2 sockets
Jun 20 23:08:17 dist-test-slave-fzjm krb5kdc[1160](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 23:08:18.904671 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:18 dist-test-slave-fzjm krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:19.015614 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43369
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:19.092638 1173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:19.092815 1173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:19.092835 1173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:19.093875 1173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:19.093909 1173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:19.093922 1173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:19.093935 1173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:19.093946 1173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:19.095332 1173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:19.095527 1173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:19.095759 1173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:19.100680 1179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:19.100684 1181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:19.100808 1173 server_base.cc:1034] running on GCE node
W20260620 23:08:19.100803 1178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:19.101181 1173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:19.101416 1173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:19.102548 1173 hybrid_clock.cc:648] HybridClock initialized: now 1781996899102488 us; error 77 us; skew 500 ppm
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:19.109308 1173 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:19.109726 1173 webserver.cc:460] Webserver started at http://127.1.32.126:33025/ using document root <none> and password file <none>
I20260620 23:08:19.109943 1173 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:19.110005 1173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:19.110157 1173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:19.111164 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "754827d418eb3d2b9021f9641a9c9120"
server_key_iv: "35f41aeeed65a9209585c31f644f914b"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.111503 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "754827d418eb3d2b9021f9641a9c9120"
server_key_iv: "35f41aeeed65a9209585c31f644f914b"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.112912 1173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:19.113622 1188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.113801 1173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:19.113899 1173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "754827d418eb3d2b9021f9641a9c9120"
server_key_iv: "35f41aeeed65a9209585c31f644f914b"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.113968 1173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:19.126381 1173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:19.128273 1173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:19.128436 1173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:19.132283 1173 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:43369
I20260620 23:08:19.132309 1240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:43369 every 8 connection(s)
I20260620 23:08:19.132620 1173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:19.133464 1241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:19.135708 1241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a: Bootstrap starting.
I20260620 23:08:19.136291 1241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:19.136560 1241 log.cc:824] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:19.137212 1241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a: No bootstrap required, opened a new log
I20260620 23:08:19.138731 1241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43369 } }
I20260620 23:08:19.138854 1241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:19.138926 1241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab75e1eb6354b9a95d4c4c66c1d210a, State: Initialized, Role: FOLLOWER
I20260620 23:08:19.139086 1241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43369 } }
I20260620 23:08:19.139145 1241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:19.139165 1241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:19.139227 1241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:19.139506 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 1173
I20260620 23:08:19.139605 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:19.139755 1241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43369 } }
I20260620 23:08:19.139854 1241 leader_election.cc:302] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ab75e1eb6354b9a95d4c4c66c1d210a; no voters:
I20260620 23:08:19.140023 1241 leader_election.cc:288] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:19.140043 1244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:19.140144 1244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [term 1 LEADER]: Becoming Leader. State: Replica: 0ab75e1eb6354b9a95d4c4c66c1d210a, State: Running, Role: LEADER
I20260620 23:08:19.140218 1244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43369 } }
I20260620 23:08:19.140267 1241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:19.140312 1153 external_mini_cluster.cc:1428] Setting key 5f620dfe32c11701ba0bd34e30b6bb0a
I20260620 23:08:19.140493 1244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43369 } } }
I20260620 23:08:19.140556 1244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:19.140744 1244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ab75e1eb6354b9a95d4c4c66c1d210a. Latest consensus state: current_term: 1 leader_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab75e1eb6354b9a95d4c4c66c1d210a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43369 } } }
I20260620 23:08:19.140813 1244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a [sys.catalog]: This master's current role is: LEADER
W20260620 23:08:19.142228 1262 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:19.142293 1262 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:19.142354 1249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:19.142555 1249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:19.144114 1249 catalog_manager.cc:1347] Generated new cluster ID: 21a474e0256e4a94a9b03bd008338832
I20260620 23:08:19.144169 1249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:19.150336 1249 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:19.150785 1249 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:19.154053 1249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ab75e1eb6354b9a95d4c4c66c1d210a: Generated new TSK 0
I20260620 23:08:19.154268 1249 catalog_manager.cc:1514] Initializing in-progress tserver states...
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:19.160897 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.141144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56346 (local address 127.1.32.126:43369)
0620 23:08:19.141563 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:19.141573 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:19.141628 (+ 55us) server_negotiation.cc:407] Connection header received
0620 23:08:19.144065 (+ 2437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:19.144072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:19.144122 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:19.144251 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:19.146229 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.147747 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.149168 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.149715 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.149827 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.157284 (+ 7457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:19.157289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:19.157289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:19.157300 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:19.159208 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.159590 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.159594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.159594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.159694 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.159939 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.159940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.159940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.160201 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:19.160339 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:19.160543 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 23:08:19.160686 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:19.185513 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43369
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:19.279753 1267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:19.279944 1267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:19.279968 1267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:19.281091 1267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:19.281136 1267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:19.281163 1267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:19.282559 1267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43369
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:19.282779 1267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:19.283053 1267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:19.288539 1272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:19.288693 1275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:19.288573 1273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:19.289063 1267 server_base.cc:1034] running on GCE node
I20260620 23:08:19.289266 1267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:19.289541 1267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:19.290709 1267 hybrid_clock.cc:648] HybridClock initialized: now 1781996899290689 us; error 31 us; skew 500 ppm
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:19.293438 1267 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:19.293886 1267 webserver.cc:460] Webserver started at http://127.1.32.65:41575/ using document root <none> and password file <none>
I20260620 23:08:19.294118 1267 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:19.294189 1267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:19.294329 1267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:19.295487 1267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "03ba2b7ab7f04723b695bb3164ef7fae"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "d9b763710503aa09f8eed4d0ef3d33a8"
server_key_iv: "90c94110ef5a239d8fca9218bd5b7c92"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.295888 1267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "03ba2b7ab7f04723b695bb3164ef7fae"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "d9b763710503aa09f8eed4d0ef3d33a8"
server_key_iv: "90c94110ef5a239d8fca9218bd5b7c92"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.297324 1267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:08:19.298133 1282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.298307 1267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:19.298415 1267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "03ba2b7ab7f04723b695bb3164ef7fae"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "d9b763710503aa09f8eed4d0ef3d33a8"
server_key_iv: "90c94110ef5a239d8fca9218bd5b7c92"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.298492 1267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:19.311436 1267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:19.313390 1267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:19.313557 1267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:19.313812 1267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:19.314142 1267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:19.314199 1267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.314248 1267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:19.314280 1267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.321113 1267 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:43387
I20260620 23:08:19.321172 1395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:43387 every 8 connection(s)
I20260620 23:08:19.321539 1267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:19.330329 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 1267
I20260620 23:08:19.330423 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:19.330575 1153 external_mini_cluster.cc:1428] Setting key f39d495b2f298023d2c4fefac5171982
I20260620 23:08:19.330964 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.322363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:37091 (local address 127.1.32.126:43369)
0620 23:08:19.322426 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:19.322428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:19.322650 (+ 222us) server_negotiation.cc:407] Connection header received
0620 23:08:19.323210 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:19.323211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:19.323227 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:19.323293 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:19.324176 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.324690 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.325675 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.325887 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.325951 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.328154 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:19.328155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:19.328155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:19.328162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:19.329472 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.329871 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.329871 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.329872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.329922 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.330176 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.330177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.330178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.330320 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:19.330395 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:19.330738 (+ 343us) server_negotiation.cc:299] Negotiation successful
0620 23:08:19.330839 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:08:19.331107 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.322339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:43369 (local address 127.1.32.65:37091)
0620 23:08:19.322523 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:19.322558 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:19.323118 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:19.323348 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:19.323351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:19.323910 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:19.324098 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:19.324102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.324778 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.324778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:19.325571 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:19.325572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.325969 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.325970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:19.326072 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.326599 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:19.326607 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:19.328070 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:19.329548 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:19.329554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:19.329556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:19.329800 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:19.329988 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:19.329988 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:19.329989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:19.330090 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:19.330420 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:19.330426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:19.330563 (+ 137us) client_negotiation.cc:769] Sending connection context
0620 23:08:19.330689 (+ 126us) client_negotiation.cc:240] Negotiation successful
0620 23:08:19.330849 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260620 23:08:19.331609 1396 heartbeater.cc:347] Connected to a master server at 127.1.32.126:43369
I20260620 23:08:19.331707 1396 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:19.331902 1396 heartbeater.cc:511] Master 127.1.32.126:43369 requested a full tablet report, sending...
I20260620 23:08:19.332470 1205 ts_manager.cc:194] Registered new tserver with Master: 03ba2b7ab7f04723b695bb3164ef7fae (127.1.32.65:43387)
I20260620 23:08:19.333180 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:37091
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:19.353657 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43369
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:19.431478 1401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:19.431694 1401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:19.431728 1401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:19.432847 1401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:19.432911 1401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:19.432957 1401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:19.434397 1401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43369
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:19.434654 1401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:19.434896 1401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:19.439664 1409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:19.439690 1407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:19.439759 1406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:19.439787 1401 server_base.cc:1034] running on GCE node
I20260620 23:08:19.440048 1401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:19.440306 1401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:19.441463 1401 hybrid_clock.cc:648] HybridClock initialized: now 1781996899441452 us; error 52 us; skew 500 ppm
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:19.444082 1401 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:19.444523 1401 webserver.cc:460] Webserver started at http://127.1.32.66:41891/ using document root <none> and password file <none>
I20260620 23:08:19.444759 1401 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:19.444830 1401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:19.444985 1401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:19.446017 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "ef47c5c9b90840528b829b590fda7714"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "e44fcb5bb1abda305e360c1620106919"
server_key_iv: "3478b23674147a4e0c08a07e6d139d64"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.446447 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "ef47c5c9b90840528b829b590fda7714"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "e44fcb5bb1abda305e360c1620106919"
server_key_iv: "3478b23674147a4e0c08a07e6d139d64"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.447887 1401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:19.448561 1416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.448724 1401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:19.448822 1401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "ef47c5c9b90840528b829b590fda7714"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "e44fcb5bb1abda305e360c1620106919"
server_key_iv: "3478b23674147a4e0c08a07e6d139d64"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.448902 1401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:19.467834 1401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:19.470506 1401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:19.470672 1401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:19.470939 1401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:19.471285 1401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:19.471342 1401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.471380 1401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:19.471411 1401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.477521 1401 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:33773
I20260620 23:08:19.477589 1529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:33773 every 8 connection(s)
I20260620 23:08:19.477901 1401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:19.486755 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.478646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:40313 (local address 127.1.32.126:43369)
0620 23:08:19.478706 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:19.478707 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:19.478933 (+ 226us) server_negotiation.cc:407] Connection header received
0620 23:08:19.479460 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:19.479462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:19.479474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:19.479536 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:19.480447 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.480945 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.481934 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.482121 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.482200 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.484202 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:19.484204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:19.484204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:19.484212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:19.485453 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.485865 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.485867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.485867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.485925 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.486158 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.486159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.486159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.486268 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:19.486340 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:19.486586 (+ 246us) server_negotiation.cc:299] Negotiation successful
0620 23:08:19.486660 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:08:19.486922 1532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.478626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:43369 (local address 127.1.32.66:40313)
0620 23:08:19.478804 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:19.478837 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:19.479356 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:19.479608 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:19.479612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:19.480162 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:19.480347 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:19.480352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.481038 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.481039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:19.481836 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:19.481838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.482209 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.482210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:19.482338 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.482787 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:19.482795 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:19.484108 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:19.485546 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:19.485552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:19.485554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:19.485779 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:19.485988 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:19.485989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:19.485989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:19.486091 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:19.486331 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:19.486337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:19.486462 (+ 125us) client_negotiation.cc:769] Sending connection context
0620 23:08:19.486556 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 23:08:19.486695 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260620 23:08:19.487411 1530 heartbeater.cc:347] Connected to a master server at 127.1.32.126:43369
I20260620 23:08:19.487505 1530 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:19.487674 1530 heartbeater.cc:511] Master 127.1.32.126:43369 requested a full tablet report, sending...
I20260620 23:08:19.487900 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 1401
I20260620 23:08:19.487995 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:19.488039 1205 ts_manager.cc:194] Registered new tserver with Master: ef47c5c9b90840528b829b590fda7714 (127.1.32.66:33773)
I20260620 23:08:19.488169 1153 external_mini_cluster.cc:1428] Setting key ce65e1719b81f01a741c263c0a3a4333
I20260620 23:08:19.488479 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:40313
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:19.511075 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43369
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:19.594480 1535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:19.594650 1535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:19.594669 1535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:19.595739 1535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:19.595778 1535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:19.595803 1535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:19.597054 1535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43369
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:19.597222 1535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:19.597373 1535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:19.602097 1541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:19.602104 1543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:19.602098 1540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:19.602365 1535 server_base.cc:1034] running on GCE node
I20260620 23:08:19.602535 1535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:19.602746 1535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:19.603878 1535 hybrid_clock.cc:648] HybridClock initialized: now 1781996899603838 us; error 56 us; skew 500 ppm
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:19.606359 1535 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:19.606803 1535 webserver.cc:460] Webserver started at http://127.1.32.67:40867/ using document root <none> and password file <none>
I20260620 23:08:19.607052 1535 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:19.607098 1535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:19.607237 1535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:19.608237 1535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "cb7b03bba5a64a43b226841ad0c0b407"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "086b9c2f1ef53060958c8838302aabb3"
server_key_iv: "c8bb5bf5ace3030af7bc0a5f042ce90e"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.608582 1535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "cb7b03bba5a64a43b226841ad0c0b407"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "086b9c2f1ef53060958c8838302aabb3"
server_key_iv: "c8bb5bf5ace3030af7bc0a5f042ce90e"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.609843 1535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:19.610494 1550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.610662 1535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:19.610761 1535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "cb7b03bba5a64a43b226841ad0c0b407"
format_stamp: "Formatted at 2026-06-20 23:08:19 on dist-test-slave-fzjm"
server_key: "086b9c2f1ef53060958c8838302aabb3"
server_key_iv: "c8bb5bf5ace3030af7bc0a5f042ce90e"
server_key_version: "encryptionkey@0"
I20260620 23:08:19.610842 1535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:19.628222 1535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:19.630093 1535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:19.630245 1535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:19.630463 1535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:19.630753 1535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:19.630801 1535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.630831 1535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:19.630856 1535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:19.636579 1535 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:46567
I20260620 23:08:19.636615 1663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:46567 every 8 connection(s)
I20260620 23:08:19.636940 1535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:19.645030 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 1535
I20260620 23:08:19.645152 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:19.645361 1153 external_mini_cluster.cc:1428] Setting key 2241b60534df1a4abfa6a2121a008199
I20260620 23:08:19.645823 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.637636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:37777 (local address 127.1.32.126:43369)
0620 23:08:19.637741 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:19.637744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:19.637887 (+ 143us) server_negotiation.cc:407] Connection header received
0620 23:08:19.638323 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:19.638324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:19.638334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:19.638379 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:19.639321 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.639891 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.640840 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.641073 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.641122 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.643083 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:19.643085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:19.643085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:19.643093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:19.644457 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.644809 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.644810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.644810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.644872 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.645134 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.645136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.645136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.645285 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:19.645357 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:19.645644 (+ 287us) server_negotiation.cc:299] Negotiation successful
0620 23:08:19.645736 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:08:19.645952 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.637610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:43369 (local address 127.1.32.67:37777)
0620 23:08:19.637776 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:19.637808 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:19.638247 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:19.638432 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:19.638435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:19.639039 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:19.639228 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:19.639233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.639964 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.639964 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:19.640777 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:19.640779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.641132 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.641133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:19.641220 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.641741 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:19.641750 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:19.643010 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:19.644533 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:19.644538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:19.644539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:19.644728 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:19.644951 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:19.644952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:19.644952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:19.645041 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:19.645354 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:19.645358 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:19.645508 (+ 150us) client_negotiation.cc:769] Sending connection context
0620 23:08:19.645587 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:08:19.645772 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260620 23:08:19.646569 1664 heartbeater.cc:347] Connected to a master server at 127.1.32.126:43369
I20260620 23:08:19.646634 1664 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:19.646745 1664 heartbeater.cc:511] Master 127.1.32.126:43369 requested a full tablet report, sending...
I20260620 23:08:19.647058 1204 ts_manager.cc:194] Registered new tserver with Master: cb7b03bba5a64a43b226841ad0c0b407 (127.1.32.67:46567)
I20260620 23:08:19.647444 1204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:37777
I20260620 23:08:19.657362 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:19.679199 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.671532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.1.32.65:43387)
0620 23:08:19.671707 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:19.671712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:19.671721 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:19.671763 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:19.671765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:19.671778 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:19.671838 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:19.672412 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.673099 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.673791 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.674079 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.674135 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.676441 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:19.676448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:19.676449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:19.676458 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:19.678157 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.678472 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.678475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.678475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.678524 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.678698 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.678699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.678699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.678826 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:19.679004 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:19.679071 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 23:08:19.679130 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
W20260620 23:08:19.679582 1310 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39728
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:19.699254 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:19.693147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39738 (local address 127.1.32.65:43387)
0620 23:08:19.693201 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:19.693202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:19.693210 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:19.693254 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:19.693255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:19.693263 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:19.693325 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:19.693793 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.694220 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.694883 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:19.695097 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:19.695142 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:19.697026 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:19.697027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:19.697027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:19.697033 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:19.698359 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.698623 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.698624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.698624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.698679 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:19.698845 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:19.698846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:19.698846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:19.698995 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:19.699075 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:19.699145 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 23:08:19.699197 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:08:19.699654 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 1267
I20260620 23:08:19.704602 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 1401
I20260620 23:08:19.709558 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 1535
I20260620 23:08:19.713907 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 1173
2026-06-20T23:08:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1819 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1678](info): set up 2 sockets
Jun 20 23:08:19 dist-test-slave-fzjm krb5kdc[1678](info): commencing operation
krb5kdc: starting...
W20260620 23:08:20.718519 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:20 dist-test-slave-fzjm krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:20.825630 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38109
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:38109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:20.901702 1691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:20.901885 1691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:20.901904 1691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:20.902985 1691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:20.903039 1691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:20.903052 1691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:20.903065 1691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:20.903120 1691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:20.904592 1691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:38109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:20.904834 1691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:20.905050 1691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:20.909435 1696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:20.909467 1697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:20.909607 1691 server_base.cc:1034] running on GCE node
W20260620 23:08:20.909511 1699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:20.909970 1691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:20.910254 1691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:20.911405 1691 hybrid_clock.cc:648] HybridClock initialized: now 1781996900911392 us; error 42 us; skew 500 ppm
Jun 20 23:08:20 dist-test-slave-fzjm krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996900, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:20.913664 1691 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:20.914048 1691 webserver.cc:460] Webserver started at http://127.1.32.126:35377/ using document root <none> and password file <none>
I20260620 23:08:20.914270 1691 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:20.914346 1691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:20.914492 1691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:20.915535 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "da2116bd82ca4e5eb6d79627823a6cf3"
format_stamp: "Formatted at 2026-06-20 23:08:20 on dist-test-slave-fzjm"
server_key: "dffc58dda03dfbbcd374dc243e73ee7d"
server_key_iv: "adcbbad57ca167b2b249a34bbb275448"
server_key_version: "encryptionkey@0"
I20260620 23:08:20.915872 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "da2116bd82ca4e5eb6d79627823a6cf3"
format_stamp: "Formatted at 2026-06-20 23:08:20 on dist-test-slave-fzjm"
server_key: "dffc58dda03dfbbcd374dc243e73ee7d"
server_key_iv: "adcbbad57ca167b2b249a34bbb275448"
server_key_version: "encryptionkey@0"
I20260620 23:08:20.917157 1691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:20.917829 1706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:20.918000 1691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:20.918098 1691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "da2116bd82ca4e5eb6d79627823a6cf3"
format_stamp: "Formatted at 2026-06-20 23:08:20 on dist-test-slave-fzjm"
server_key: "dffc58dda03dfbbcd374dc243e73ee7d"
server_key_iv: "adcbbad57ca167b2b249a34bbb275448"
server_key_version: "encryptionkey@0"
I20260620 23:08:20.918166 1691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:20.932804 1691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:20.934619 1691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:20.934787 1691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:20.938580 1691 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:38109
I20260620 23:08:20.938629 1758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:38109 every 8 connection(s)
I20260620 23:08:20.939126 1691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:20.939605 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 1691
I20260620 23:08:20.939675 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:20.939862 1153 external_mini_cluster.cc:1428] Setting key f5d672f78a17d196f95ef60e1459c457
I20260620 23:08:20.940205 1759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:20.943511 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3: Bootstrap starting.
I20260620 23:08:20.944126 1759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:20.944454 1759 log.cc:824] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:20.945245 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3: No bootstrap required, opened a new log
Jun 20 23:08:20 dist-test-slave-fzjm krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:20.947933 1759 raft_consensus.cc:348] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38109 } }
I20260620 23:08:20.948100 1759 raft_consensus.cc:374] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:20.948153 1759 raft_consensus.cc:729] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2116bd82ca4e5eb6d79627823a6cf3, State: Initialized, Role: FOLLOWER
I20260620 23:08:20.948283 1759 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38109 } }
I20260620 23:08:20.948374 1759 raft_consensus.cc:388] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:20.948412 1759 raft_consensus.cc:482] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:20.948470 1759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:20.949174 1759 raft_consensus.cc:504] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38109 } }
I20260620 23:08:20.949283 1759 leader_election.cc:302] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2116bd82ca4e5eb6d79627823a6cf3; no voters:
I20260620 23:08:20.949432 1759 leader_election.cc:288] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:20.949505 1764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:20.949669 1764 raft_consensus.cc:686] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [term 1 LEADER]: Becoming Leader. State: Replica: da2116bd82ca4e5eb6d79627823a6cf3, State: Running, Role: LEADER
I20260620 23:08:20.949687 1759 sys_catalog.cc:564] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:20.949765 1764 consensus_queue.cc:237] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38109 } }
I20260620 23:08:20.950160 1764 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da2116bd82ca4e5eb6d79627823a6cf3. Latest consensus state: current_term: 1 leader_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38109 } } }
I20260620 23:08:20.950275 1764 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:20.950160 1766 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2116bd82ca4e5eb6d79627823a6cf3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38109 } } }
I20260620 23:08:20.950425 1766 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:20.951115 1772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:20.951890 1772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:20.951911 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:20.940423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.1.32.126:38109)
0620 23:08:20.940621 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:20.940631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:20.940663 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:08:20.941321 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:20.941330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:20.941395 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:20.941552 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:20.942095 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:20.943463 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:20.944246 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:20.944836 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:20.944927 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:20.947619 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:20.947624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:20.947625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:20.947637 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:20.949658 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:20.950111 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:20.950115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:20.950115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:20.950249 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:20.950505 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:20.950506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:20.950506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:20.950838 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:20.951452 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:20.951539 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 23:08:20.951704 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260620 23:08:20.953392 1772 catalog_manager.cc:1347] Generated new cluster ID: 119486f13dcb441dbb0b5a0b5a018de5
I20260620 23:08:20.953470 1772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:20.961670 1772 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:20.962144 1772 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:20.970606 1772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da2116bd82ca4e5eb6d79627823a6cf3: Generated new TSK 0
I20260620 23:08:20.970789 1772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:21.003577 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38109
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:21.095938 1785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:21.096130 1785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:21.096153 1785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:21.097188 1785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:21.097220 1785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:21.097250 1785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:21.098503 1785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:21.098665 1785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:21.098826 1785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:08:21.103579 1785 server_base.cc:1034] running on GCE node
W20260620 23:08:21.103554 1793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:21.103550 1790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:21.103539 1791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:21.104045 1785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:21.104290 1785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:21.105430 1785 hybrid_clock.cc:648] HybridClock initialized: now 1781996901105412 us; error 33 us; skew 500 ppm
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:21.107913 1785 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:21.108316 1785 webserver.cc:460] Webserver started at http://127.1.32.65:39701/ using document root <none> and password file <none>
I20260620 23:08:21.108525 1785 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:21.108566 1785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:21.108718 1785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:21.109755 1785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "1934aa0cad3242b386c30156d487bc65"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "9ff4d3c1a26206fc92a3f0d480876b7b"
server_key_iv: "ab6f458f9025f5e6877dd6296c6235fe"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.110091 1785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "1934aa0cad3242b386c30156d487bc65"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "9ff4d3c1a26206fc92a3f0d480876b7b"
server_key_iv: "ab6f458f9025f5e6877dd6296c6235fe"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.112131 1785 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 23:08:21.112926 1800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.113106 1785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:21.113178 1785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "1934aa0cad3242b386c30156d487bc65"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "9ff4d3c1a26206fc92a3f0d480876b7b"
server_key_iv: "ab6f458f9025f5e6877dd6296c6235fe"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.113277 1785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:21.126047 1785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:21.127839 1785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:21.127987 1785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:21.128189 1785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:21.128480 1785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:21.128513 1785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.128533 1785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:21.128578 1785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.134188 1785 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:33539
I20260620 23:08:21.134236 1913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:33539 every 8 connection(s)
I20260620 23:08:21.134507 1785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:21.138500 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 1785
I20260620 23:08:21.138599 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:21.138747 1153 external_mini_cluster.cc:1428] Setting key b5def9eb88482cd6b889dafeaaad4151
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:21.144119 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.135135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38109 (local address 127.1.32.65:41977)
0620 23:08:21.135306 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.135350 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.136233 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.136465 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.136468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.136696 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:21.136873 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.136877 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.137506 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.137506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.138270 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.138272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.138707 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.138708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.138842 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.139303 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:21.139312 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:21.140958 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:21.142734 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:21.142739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:21.142741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:21.143045 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:21.143275 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:21.143276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:21.143276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:21.143369 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:21.143659 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:21.143663 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:21.143782 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.143861 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.143972 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260620 23:08:21.144161 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.135151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:41977 (local address 127.1.32.126:38109)
0620 23:08:21.135196 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.135198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.135421 (+ 223us) server_negotiation.cc:407] Connection header received
0620 23:08:21.136344 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.136345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.136355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.136408 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:21.136943 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.137426 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.138361 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.138576 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.138676 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.141062 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:21.141063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:21.141064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:21.141072 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:21.142645 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.143137 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.143139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.143139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.143202 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.143451 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.143452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.143453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.143592 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:21.143668 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.143911 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.144021 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:08:21.144829 1914 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38109
I20260620 23:08:21.144933 1914 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:21.145090 1914 heartbeater.cc:511] Master 127.1.32.126:38109 requested a full tablet report, sending...
I20260620 23:08:21.145627 1723 ts_manager.cc:194] Registered new tserver with Master: 1934aa0cad3242b386c30156d487bc65 (127.1.32.65:33539)
I20260620 23:08:21.146241 1723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:41977
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:21.162103 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38109
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:21.239288 1919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:21.239518 1919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:21.239555 1919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:21.240725 1919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:21.240795 1919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:21.240839 1919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:21.242336 1919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:21.242588 1919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:21.242816 1919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:21.247951 1924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:21.247967 1927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:21.247959 1925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:21.248134 1919 server_base.cc:1034] running on GCE node
I20260620 23:08:21.248327 1919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:21.248545 1919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:21.249675 1919 hybrid_clock.cc:648] HybridClock initialized: now 1781996901249665 us; error 26 us; skew 500 ppm
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:21.252314 1919 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:21.252749 1919 webserver.cc:460] Webserver started at http://127.1.32.66:32895/ using document root <none> and password file <none>
I20260620 23:08:21.252966 1919 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:21.253032 1919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:21.253181 1919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:21.254186 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "6273a9f566734eed95f400850277baa6"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "a1a632c06db7ab413cac90dd283bfaae"
server_key_iv: "ce0792344e55a00398d47dab7271e462"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.254542 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "6273a9f566734eed95f400850277baa6"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "a1a632c06db7ab413cac90dd283bfaae"
server_key_iv: "ce0792344e55a00398d47dab7271e462"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.255946 1919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260620 23:08:21.256647 1934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.256837 1919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:21.256934 1919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "6273a9f566734eed95f400850277baa6"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "a1a632c06db7ab413cac90dd283bfaae"
server_key_iv: "ce0792344e55a00398d47dab7271e462"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.257019 1919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:21.268592 1919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:21.270650 1919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:21.270821 1919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:21.271063 1919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:21.271347 1919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:21.271404 1919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.271440 1919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:21.271483 1919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.276847 1919 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:41435
I20260620 23:08:21.276916 2047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:41435 every 8 connection(s)
I20260620 23:08:21.277215 1919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:21.286347 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 1919
I20260620 23:08:21.286417 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:21.286585 1153 external_mini_cluster.cc:1428] Setting key 8b8c18ea479d816b1686baf70211d084
I20260620 23:08:21.286674 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.277992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:46981 (local address 127.1.32.126:38109)
0620 23:08:21.278055 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.278058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.278241 (+ 183us) server_negotiation.cc:407] Connection header received
0620 23:08:21.279309 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.279311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.279325 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.279376 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:21.280020 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.280530 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.281523 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.281730 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.281836 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.284220 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:21.284221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:21.284222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:21.284229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:21.285336 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.285766 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.285768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.285768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.285826 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.286060 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.286061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.286062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.286168 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:21.286242 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.286509 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.286590 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:08:21.286928 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.277971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38109 (local address 127.1.32.66:46981)
0620 23:08:21.278124 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.278157 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.279187 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.279463 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.279466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.279712 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:21.279906 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.279910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.280625 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.280626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.281428 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.281430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.281843 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.281844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.281961 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.282437 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:21.282445 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:21.284133 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:21.285447 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:21.285453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:21.285455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:21.285676 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:21.285889 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:21.285890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:21.285890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:21.285995 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:21.286231 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:21.286236 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:21.286369 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.286479 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.286624 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260620 23:08:21.287370 2048 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38109
I20260620 23:08:21.287456 2048 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:21.287637 2048 heartbeater.cc:511] Master 127.1.32.126:38109 requested a full tablet report, sending...
I20260620 23:08:21.287971 1723 ts_manager.cc:194] Registered new tserver with Master: 6273a9f566734eed95f400850277baa6 (127.1.32.66:41435)
I20260620 23:08:21.288292 1723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:46981
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:21.309805 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38109
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:21.388447 2053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:21.388636 2053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:21.388659 2053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:21.389840 2053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:21.389880 2053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:21.389907 2053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:21.391345 2053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38109
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:21.391557 2053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:21.391784 2053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:21.396595 2058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:21.396611 2059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:21.396636 2053 server_base.cc:1034] running on GCE node
W20260620 23:08:21.396633 2061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:21.397003 2053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:21.397182 2053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:21.398350 2053 hybrid_clock.cc:648] HybridClock initialized: now 1781996901398344 us; error 34 us; skew 500 ppm
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:21.400682 2053 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:21.401149 2053 webserver.cc:460] Webserver started at http://127.1.32.67:46201/ using document root <none> and password file <none>
I20260620 23:08:21.401356 2053 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:21.401413 2053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:21.401577 2053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:21.402482 2053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "ec67528ef18d43af81a61fcd05f05556"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "5c501f48c18c357a89558df12b7a5f55"
server_key_iv: "733b0da0ef179f021a2c8b2899e373a8"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.402806 2053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "ec67528ef18d43af81a61fcd05f05556"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "5c501f48c18c357a89558df12b7a5f55"
server_key_iv: "733b0da0ef179f021a2c8b2899e373a8"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.404116 2053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:21.404829 2068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.405009 2053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:21.405092 2053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "ec67528ef18d43af81a61fcd05f05556"
format_stamp: "Formatted at 2026-06-20 23:08:21 on dist-test-slave-fzjm"
server_key: "5c501f48c18c357a89558df12b7a5f55"
server_key_iv: "733b0da0ef179f021a2c8b2899e373a8"
server_key_version: "encryptionkey@0"
I20260620 23:08:21.405146 2053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:21.426402 2053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:21.428134 2053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:21.428244 2053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:21.428413 2053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:21.428699 2053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:21.428730 2053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.428751 2053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:21.428799 2053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:21.434142 2053 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:46335
I20260620 23:08:21.434229 2181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:46335 every 8 connection(s)
I20260620 23:08:21.434507 2053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:21.442567 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.435189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:36785 (local address 127.1.32.126:38109)
0620 23:08:21.435256 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.435258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.435438 (+ 180us) server_negotiation.cc:407] Connection header received
0620 23:08:21.436286 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.436287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.436296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.436344 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:21.436912 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.437372 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.438315 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.438488 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.438535 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.440448 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:21.440449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:21.440449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:21.440455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:21.441538 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.441886 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.441887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.441887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.441933 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.442121 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.442122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.442122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.442212 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:21.442258 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.442456 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.442509 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:08:21.442771 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.435154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38109 (local address 127.1.32.67:36785)
0620 23:08:21.435320 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.435357 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.436201 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.436398 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.436401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.436662 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:21.436845 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.436849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.437442 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.437442 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.438233 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.438235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.438552 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.438553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.438638 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.439167 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:21.439174 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:21.440370 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:21.441603 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:21.441608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:21.441610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:21.441811 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:21.441986 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:21.441987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:21.441987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:21.442071 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:21.442261 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:21.442264 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:21.442359 (+ 95us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.442432 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.442535 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260620 23:08:21.443262 2182 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38109
I20260620 23:08:21.443353 2182 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:21.443508 2182 heartbeater.cc:511] Master 127.1.32.126:38109 requested a full tablet report, sending...
I20260620 23:08:21.443893 1723 ts_manager.cc:194] Registered new tserver with Master: ec67528ef18d43af81a61fcd05f05556 (127.1.32.67:46335)
I20260620 23:08:21.444061 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 2053
I20260620 23:08:21.444156 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:21.444342 1153 external_mini_cluster.cc:1428] Setting key 767a3562eba61f50a37fa7db0150757f
I20260620 23:08:21.444343 1723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:36785
I20260620 23:08:21.445103 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:08:21.452579 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.446561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.1.32.126:38109)
0620 23:08:21.446595 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.446596 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.446739 (+ 143us) server_negotiation.cc:407] Connection header received
0620 23:08:21.446798 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.446799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.446805 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.446847 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:21.447444 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.447891 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.448619 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.448812 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.448867 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.450137 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:21.450138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:21.450138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:21.450143 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:21.451610 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.451933 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.451933 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.451934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.451979 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.452150 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.452150 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.452150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.452283 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:21.452354 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.452469 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.452525 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:21.454605 1723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:21.456220 1723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:21.462550 2191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.458438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:46335 (local address 127.0.0.1:40362)
0620 23:08:21.459128 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.459139 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.459189 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.459395 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.459396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.459398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:21.459445 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.459447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.460455 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.460456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.461507 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.461509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.462264 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.462265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.462342 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.462354 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.462429 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.462477 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":51,"threads_started":1}
I20260620 23:08:21.462569 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.458401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33520 (local address 127.1.32.65:33539)
0620 23:08:21.458542 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.458546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.458755 (+ 209us) server_negotiation.cc:407] Connection header received
0620 23:08:21.458834 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.458835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.458865 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.459016 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:21.459159 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.460052 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.461551 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.462227 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.462306 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.462357 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.462453 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.462492 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260620 23:08:21.462821 2193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.458248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:33539 (local address 127.0.0.1:33520)
0620 23:08:21.458673 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.458687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.458762 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.459016 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.459018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.459020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:21.459089 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.459091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.460138 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.460139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.461446 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.461449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.462308 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.462309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.462368 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.462373 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.462714 (+ 341us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.462758 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":36,"thread_start_us":48,"threads_started":1}
I20260620 23:08:21.462565 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.458514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.1.32.67:46335)
0620 23:08:21.458658 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.458661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.459251 (+ 590us) server_negotiation.cc:407] Connection header received
0620 23:08:21.459301 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.459303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.459318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.459397 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:21.459512 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.460337 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.461602 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.462107 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.462178 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.462228 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.462453 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.462492 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260620 23:08:21.463479 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.458243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41435 (local address 127.0.0.1:34040)
0620 23:08:21.458567 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.458603 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.458667 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.459029 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.459030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.459032 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:21.459096 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.459098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.460888 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.460889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.461986 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.461988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.463216 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.463217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.463289 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.463293 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.463354 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.463388 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":49,"threads_started":1}
I20260620 23:08:21.463539 2116 tablet_service.cc:1467] Processing CreateTablet for tablet 7f844122291149388722a919c41518f7 (DEFAULT_TABLE table=test-table [id=0a33126ae0504fdaad050e1532808585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:21.463511 1848 tablet_service.cc:1467] Processing CreateTablet for tablet 7f844122291149388722a919c41518f7 (DEFAULT_TABLE table=test-table [id=0a33126ae0504fdaad050e1532808585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:21.463876 1848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f844122291149388722a919c41518f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:21.463910 2116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f844122291149388722a919c41518f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:21.464020 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.458575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.1.32.66:41435)
0620 23:08:21.458810 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.458820 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.458831 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:21.458869 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.458871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.458923 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.459023 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:21.460143 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.460794 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.462098 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.462822 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.462941 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.463002 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.463779 (+ 777us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.463846 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":72,"threads_started":1}
I20260620 23:08:21.464915 1982 tablet_service.cc:1467] Processing CreateTablet for tablet 7f844122291149388722a919c41518f7 (DEFAULT_TABLE table=test-table [id=0a33126ae0504fdaad050e1532808585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:21.465215 1982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f844122291149388722a919c41518f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:21.466478 2196 tablet_bootstrap.cc:492] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65: Bootstrap starting.
I20260620 23:08:21.467042 2196 tablet_bootstrap.cc:654] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:21.467343 2197 tablet_bootstrap.cc:492] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556: Bootstrap starting.
I20260620 23:08:21.467581 2196 log.cc:824] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:21.467927 2197 tablet_bootstrap.cc:654] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:21.468304 2199 tablet_bootstrap.cc:492] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6: Bootstrap starting.
I20260620 23:08:21.468403 2196 tablet_bootstrap.cc:492] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65: No bootstrap required, opened a new log
I20260620 23:08:21.468474 2196 ts_tablet_manager.cc:1397] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:21.468492 2197 log.cc:824] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:21.468878 2199 tablet_bootstrap.cc:654] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:21.469352 2199 log.cc:824] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:21.470052 2199 tablet_bootstrap.cc:492] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6: No bootstrap required, opened a new log
I20260620 23:08:21.470108 2199 ts_tablet_manager.cc:1397] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:21.470137 2196 raft_consensus.cc:348] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.470302 2196 raft_consensus.cc:374] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:21.470345 2196 raft_consensus.cc:729] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1934aa0cad3242b386c30156d487bc65, State: Initialized, Role: FOLLOWER
I20260620 23:08:21.470451 2196 consensus_queue.cc:260] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.470754 2196 ts_tablet_manager.cc:1428] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:08:21.470762 1914 heartbeater.cc:503] Master 127.1.32.126:38109 was elected leader, sending a full tablet report...
I20260620 23:08:21.471431 2197 tablet_bootstrap.cc:492] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556: No bootstrap required, opened a new log
I20260620 23:08:21.471508 2197 ts_tablet_manager.cc:1397] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 23:08:21.472214 2199 raft_consensus.cc:348] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.472368 2199 raft_consensus.cc:374] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:21.472390 2199 raft_consensus.cc:729] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6273a9f566734eed95f400850277baa6, State: Initialized, Role: FOLLOWER
I20260620 23:08:21.472520 2199 consensus_queue.cc:260] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.472853 2199 ts_tablet_manager.cc:1428] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:21.472903 2048 heartbeater.cc:503] Master 127.1.32.126:38109 was elected leader, sending a full tablet report...
I20260620 23:08:21.473599 2197 raft_consensus.cc:348] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.473798 2197 raft_consensus.cc:374] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:21.473845 2197 raft_consensus.cc:729] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec67528ef18d43af81a61fcd05f05556, State: Initialized, Role: FOLLOWER
I20260620 23:08:21.473984 2197 consensus_queue.cc:260] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.474228 2197 ts_tablet_manager.cc:1428] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:21.474291 2182 heartbeater.cc:503] Master 127.1.32.126:38109 was elected leader, sending a full tablet report...
I20260620 23:08:21.523649 2204 raft_consensus.cc:482] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:21.523813 2204 raft_consensus.cc:504] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.524204 2204 leader_election.cc:288] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1934aa0cad3242b386c30156d487bc65 (127.1.32.65:33539), 6273a9f566734eed95f400850277baa6 (127.1.32.66:41435)
I20260620 23:08:21.527109 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.524540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:47077 (local address 127.1.32.65:33539)
0620 23:08:21.524610 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.524612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.524621 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:21.524647 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.524648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.524661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.524759 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:21.524924 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.525409 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.526348 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.526725 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.526805 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.526816 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.527008 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.527042 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:08:21.527109 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.524363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:33539 (local address 127.1.32.67:47077)
0620 23:08:21.524404 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.524416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.524466 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.524765 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.524765 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.524767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:21.524835 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.524837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.525493 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.525494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.526256 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.526258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.526816 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.526817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.526948 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.526952 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.527003 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.527042 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 23:08:21.527208 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.524455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:52463 (local address 127.1.32.66:41435)
0620 23:08:21.524516 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.524519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.524795 (+ 276us) server_negotiation.cc:407] Connection header received
0620 23:08:21.524890 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.524892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.524903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.524980 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:21.525121 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.525583 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.526476 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.526842 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.526946 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.526956 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.527101 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.527133 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:08:21.527410 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.524363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41435 (local address 127.1.32.67:52463)
0620 23:08:21.524726 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:21.524736 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:21.524810 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:21.524985 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:21.524986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:21.524987 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:21.525059 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.525060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.525667 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.525668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.526392 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:21.526394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.526948 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.526949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:21.527027 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.527030 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:21.527095 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:08:21.527132 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":154,"thread_start_us":133,"threads_started":1}
I20260620 23:08:21.527418 1868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f844122291149388722a919c41518f7" candidate_uuid: "ec67528ef18d43af81a61fcd05f05556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1934aa0cad3242b386c30156d487bc65" is_pre_election: true
I20260620 23:08:21.527556 1868 raft_consensus.cc:2454] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec67528ef18d43af81a61fcd05f05556 in term 0.
I20260620 23:08:21.527765 2002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f844122291149388722a919c41518f7" candidate_uuid: "ec67528ef18d43af81a61fcd05f05556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6273a9f566734eed95f400850277baa6" is_pre_election: true
W20260620 23:08:21.527864 2049 tablet.cc:2369] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:21.527911 2002 raft_consensus.cc:2454] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec67528ef18d43af81a61fcd05f05556 in term 0.
I20260620 23:08:21.528194 2069 leader_election.cc:302] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6273a9f566734eed95f400850277baa6, ec67528ef18d43af81a61fcd05f05556; no voters:
I20260620 23:08:21.528312 2204 raft_consensus.cc:2781] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:21.528367 2204 raft_consensus.cc:482] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:21.528393 2204 raft_consensus.cc:3037] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:21.529017 2204 raft_consensus.cc:504] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.529162 2204 leader_election.cc:288] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [CANDIDATE]: Term 1 election: Requested vote from peers 1934aa0cad3242b386c30156d487bc65 (127.1.32.65:33539), 6273a9f566734eed95f400850277baa6 (127.1.32.66:41435)
I20260620 23:08:21.529310 2002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f844122291149388722a919c41518f7" candidate_uuid: "ec67528ef18d43af81a61fcd05f05556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6273a9f566734eed95f400850277baa6"
I20260620 23:08:21.529305 1868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f844122291149388722a919c41518f7" candidate_uuid: "ec67528ef18d43af81a61fcd05f05556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1934aa0cad3242b386c30156d487bc65"
I20260620 23:08:21.529404 2002 raft_consensus.cc:3037] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:21.529402 1868 raft_consensus.cc:3037] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:21.530172 1868 raft_consensus.cc:2454] T 7f844122291149388722a919c41518f7 P 1934aa0cad3242b386c30156d487bc65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec67528ef18d43af81a61fcd05f05556 in term 1.
I20260620 23:08:21.530205 2002 raft_consensus.cc:2454] T 7f844122291149388722a919c41518f7 P 6273a9f566734eed95f400850277baa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec67528ef18d43af81a61fcd05f05556 in term 1.
I20260620 23:08:21.530364 2072 leader_election.cc:302] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1934aa0cad3242b386c30156d487bc65, ec67528ef18d43af81a61fcd05f05556; no voters:
I20260620 23:08:21.530526 2204 raft_consensus.cc:2781] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:21.530697 2204 raft_consensus.cc:686] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [term 1 LEADER]: Becoming Leader. State: Replica: ec67528ef18d43af81a61fcd05f05556, State: Running, Role: LEADER
I20260620 23:08:21.530797 2204 consensus_queue.cc:237] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } }
I20260620 23:08:21.531615 1723 catalog_manager.cc:5495] T 7f844122291149388722a919c41518f7 P ec67528ef18d43af81a61fcd05f05556 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec67528ef18d43af81a61fcd05f05556 (127.1.32.67). New cstate: current_term: 1 leader_uuid: "ec67528ef18d43af81a61fcd05f05556" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1934aa0cad3242b386c30156d487bc65" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec67528ef18d43af81a61fcd05f05556" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6273a9f566734eed95f400850277baa6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41435 } health_report { overall_health: UNKNOWN } } }
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:21.556740 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.550060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33528 (local address 127.1.32.65:33539)
0620 23:08:21.550118 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.550121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.550275 (+ 154us) server_negotiation.cc:407] Connection header received
0620 23:08:21.550314 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.550315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.550335 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.550373 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:21.550858 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.551369 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.552014 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.552185 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.552235 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.554474 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:21.554483 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:21.554483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:21.554489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:21.555832 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.556102 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.556104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.556104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.556151 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.556319 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.556320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.556320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.556436 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:21.556531 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.556584 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.556639 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:21.578653 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.572372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33536 (local address 127.1.32.65:33539)
0620 23:08:21.572420 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.572423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.572444 (+ 21us) server_negotiation.cc:407] Connection header received
0620 23:08:21.572467 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.572467 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.572476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.572523 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:21.573015 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.573469 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.574129 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.574307 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.574359 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.576411 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:21.576412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:21.576412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:21.576418 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:21.577615 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.577951 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.577952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.577953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.578004 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.578234 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.578235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.578235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.578319 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:21.578387 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.578521 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.578577 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:21.598423 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:21.592335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.1.32.65:33539)
0620 23:08:21.592394 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:21.592395 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:21.592407 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:08:21.592470 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:21.592471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:21.592479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:21.592526 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:21.593028 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.593545 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.594260 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:21.594431 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:21.594483 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:21.596463 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:21.596465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:21.596465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:21.596470 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:21.597549 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.597845 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.597846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.597846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.597891 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:21.598035 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:21.598035 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:21.598035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:21.598112 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:21.598177 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:21.598274 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 23:08:21.598323 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 23:08:21.598879 1828 tablet_service.cc:2908] Rejecting scan request for tablet 7f844122291149388722a919c41518f7: Uninitialized: clean time has not yet been initialized
I20260620 23:08:21.599807 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 1785
I20260620 23:08:21.604493 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 1919
I20260620 23:08:21.609174 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 2053
I20260620 23:08:21.613441 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 1691
2026-06-20T23:08:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1900 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[2220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[2220](info): set up 2 sockets
Jun 20 23:08:21 dist-test-slave-fzjm krb5kdc[2220](info): commencing operation
krb5kdc: starting...
W20260620 23:08:22.626485 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:22 dist-test-slave-fzjm krb5kdc[2220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:22.739500 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45985
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:45985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:22.819950 2233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:22.820190 2233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:22.820230 2233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:22.821542 2233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:22.821604 2233 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 23:08:22.821630 2233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:22.821662 2233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:22.821712 2233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:22.821738 2233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:22.823184 2233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:45985
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:22.823452 2233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:22.823697 2233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:22.828454 2241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:22.828464 2238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:22.828507 2233 server_base.cc:1034] running on GCE node
W20260620 23:08:22.828465 2239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:22.828920 2233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:22.829200 2233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:22.830370 2233 hybrid_clock.cc:648] HybridClock initialized: now 1781996902830357 us; error 28 us; skew 500 ppm
Jun 20 23:08:22 dist-test-slave-fzjm krb5kdc[2220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996902, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:22.832830 2233 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:22.833262 2233 webserver.cc:460] Webserver started at http://127.1.32.126:43587/ using document root <none> and password file <none>
I20260620 23:08:22.833436 2233 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:22.833477 2233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:22.833562 2233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:22.834558 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "3eda54aa7e644bab939117dd70e89c7c"
format_stamp: "Formatted at 2026-06-20 23:08:22 on dist-test-slave-fzjm"
server_key: "a6443e0663f2df1f997b265cc4345687"
server_key_iv: "913ed5cccc43c8903ce1bc8dd453b3ad"
server_key_version: "encryptionkey@0"
I20260620 23:08:22.834856 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "3eda54aa7e644bab939117dd70e89c7c"
format_stamp: "Formatted at 2026-06-20 23:08:22 on dist-test-slave-fzjm"
server_key: "a6443e0663f2df1f997b265cc4345687"
server_key_iv: "913ed5cccc43c8903ce1bc8dd453b3ad"
server_key_version: "encryptionkey@0"
I20260620 23:08:22.836160 2233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:22.836855 2248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:22.837061 2233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:22.837167 2233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "3eda54aa7e644bab939117dd70e89c7c"
format_stamp: "Formatted at 2026-06-20 23:08:22 on dist-test-slave-fzjm"
server_key: "a6443e0663f2df1f997b265cc4345687"
server_key_iv: "913ed5cccc43c8903ce1bc8dd453b3ad"
server_key_version: "encryptionkey@0"
I20260620 23:08:22.837244 2233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:22.856532 2233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:22.859167 2233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:22.859338 2233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:22.864144 2233 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:45985
I20260620 23:08:22.864221 2310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:45985 every 8 connection(s)
I20260620 23:08:22.864573 2233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:22.865262 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:22.867543 2311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c: Bootstrap starting.
I20260620 23:08:22.868091 2311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:22.868382 2311 log.cc:824] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:22.869000 2311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c: No bootstrap required, opened a new log
I20260620 23:08:22.871002 2311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eda54aa7e644bab939117dd70e89c7c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45985 } }
I20260620 23:08:22.871131 2311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:22.871153 2311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eda54aa7e644bab939117dd70e89c7c, State: Initialized, Role: FOLLOWER
I20260620 23:08:22.871325 2311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eda54aa7e644bab939117dd70e89c7c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45985 } }
I20260620 23:08:22.871409 2311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:22.871459 2311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:22.871508 2311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:22.872028 2311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eda54aa7e644bab939117dd70e89c7c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45985 } }
I20260620 23:08:22.872110 2311 leader_election.cc:302] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eda54aa7e644bab939117dd70e89c7c; no voters:
I20260620 23:08:22.872278 2311 leader_election.cc:288] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:22.872414 2314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:22.872573 2311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:22.872578 2314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [term 1 LEADER]: Becoming Leader. State: Replica: 3eda54aa7e644bab939117dd70e89c7c, State: Running, Role: LEADER
I20260620 23:08:22.872713 2314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eda54aa7e644bab939117dd70e89c7c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45985 } }
I20260620 23:08:22.873044 2315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eda54aa7e644bab939117dd70e89c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eda54aa7e644bab939117dd70e89c7c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45985 } } }
I20260620 23:08:22.873045 2316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eda54aa7e644bab939117dd70e89c7c. Latest consensus state: current_term: 1 leader_uuid: "3eda54aa7e644bab939117dd70e89c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eda54aa7e644bab939117dd70e89c7c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45985 } } }
I20260620 23:08:22.873144 2315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:22.873150 2316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:22.873413 2323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:22.873735 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 2233
I20260620 23:08:22.873835 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:22.873993 1153 external_mini_cluster.cc:1428] Setting key 8c6e142c49d8f535b3510c76ee1e7cad
I20260620 23:08:22.874015 2323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:22.875864 2323 catalog_manager.cc:1347] Generated new cluster ID: 1c85172bd79a464289cee84370ff8a5b
I20260620 23:08:22.875918 2323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:22 dist-test-slave-fzjm krb5kdc[2220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:22.885210 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:22.874422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55864 (local address 127.1.32.126:45985)
0620 23:08:22.874583 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:22.874592 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:22.874627 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:08:22.874947 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:22.874955 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:22.875015 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:22.875145 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:22.875930 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:22.877258 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:22.878355 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:22.878964 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:22.879055 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:22.881730 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:22.881735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:22.881735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:22.881747 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:22.883672 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:22.884068 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:22.884071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:22.884071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:22.884170 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:22.884358 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:22.884359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:22.884359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:22.884650 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:22.884805 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:22.884878 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:08:22.885011 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260620 23:08:22.886108 2323 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:22.886531 2323 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:22.889281 2323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c: Generated new TSK 0
I20260620 23:08:22.889410 2323 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:22.914347 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45985
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:22.992619 2337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:22.992854 2337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:22.992892 2337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:22.994087 2337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:22.994151 2337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:22.994185 2337 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:22.994225 2337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:22.995677 2337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:45985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:22.995946 2337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:22.996170 2337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:23.001184 2343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:23.001188 2345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:23.001237 2342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:23.001228 2337 server_base.cc:1034] running on GCE node
I20260620 23:08:23.001610 2337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:23.001842 2337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:23.002946 2337 hybrid_clock.cc:648] HybridClock initialized: now 1781996903002926 us; error 30 us; skew 500 ppm
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:23.005388 2337 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:23.005810 2337 webserver.cc:460] Webserver started at http://127.1.32.65:43061/ using document root <none> and password file <none>
I20260620 23:08:23.006027 2337 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:23.006089 2337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:23.006244 2337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:23.007336 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "7f918fa8bd8b429186675e60017c2b0f"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "a5990a42357f6ad38a32f2326e4fc906"
server_key_iv: "badc0a81778b09729b78bb158a282757"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.007678 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "7f918fa8bd8b429186675e60017c2b0f"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "a5990a42357f6ad38a32f2326e4fc906"
server_key_iv: "badc0a81778b09729b78bb158a282757"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.008910 2337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:23.009580 2352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.009742 2337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:23.009833 2337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "7f918fa8bd8b429186675e60017c2b0f"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "a5990a42357f6ad38a32f2326e4fc906"
server_key_iv: "badc0a81778b09729b78bb158a282757"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.009907 2337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:23.050511 2337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:23.052603 2337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:23.052783 2337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:23.053328 2337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:23.053396 2337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.053442 2337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:23.053474 2337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:23.061004 2337 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:35879
I20260620 23:08:23.061380 2337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:23.061482 2467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:35879 every 8 connection(s)
I20260620 23:08:23.063639 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.053842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:58329 (local address 127.1.32.126:45985)
0620 23:08:23.053910 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.053913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.054129 (+ 216us) server_negotiation.cc:407] Connection header received
0620 23:08:23.054616 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.054618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.054660 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.054758 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.055466 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.056276 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.057357 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.057722 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.057795 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.060544 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:23.060546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:23.060546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:23.060555 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:23.062154 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.062625 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.062627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.062627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.062684 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.062970 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.062971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.062972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.063085 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:23.063155 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.063432 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.063547 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:08:23.063797 2361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.053842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45985 (local address 127.1.32.65:58329)
0620 23:08:23.053999 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.054039 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.054521 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.054866 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.054870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.055171 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:23.055382 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.055387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.056407 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.056408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.057260 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.057262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.057804 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.057805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.057967 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.058467 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:23.058493 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:23.060458 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:23.062252 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.062259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.062261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.062557 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.062761 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.062762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.062763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.062872 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.063156 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:23.063164 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:23.063294 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.063397 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.063575 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260620 23:08:23.064363 2468 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45985
I20260620 23:08:23.064455 2468 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:23.064692 2468 heartbeater.cc:511] Master 127.1.32.126:45985 requested a full tablet report, sending...
I20260620 23:08:23.065189 2265 ts_manager.cc:194] Registered new tserver with Master: 7f918fa8bd8b429186675e60017c2b0f (127.1.32.65:35879)
I20260620 23:08:23.066046 2265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:58329
I20260620 23:08:23.069512 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 2337
I20260620 23:08:23.069620 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:23.069801 1153 external_mini_cluster.cc:1428] Setting key 8fb320681f5540f9a018d8184465e32c
I20260620 23:08:23.070652 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.065424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45985 (local address 127.1.32.65:54951)
0620 23:08:23.065539 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.065548 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.065605 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.065724 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.065724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.065867 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:23.065914 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.065915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.066614 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.066615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.067236 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.067238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.067598 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.067599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.067675 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.068022 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:23.068026 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:23.068275 (+ 249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:23.069585 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.069586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.069587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.069786 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.070067 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.070068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.070068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.070139 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.070381 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:23.070381 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:23.070429 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.070502 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.070592 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
I20260620 23:08:23.070739 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.065488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:54951 (local address 127.1.32.126:45985)
0620 23:08:23.065528 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.065529 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.065594 (+ 65us) server_negotiation.cc:407] Connection header received
0620 23:08:23.065657 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.065658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.065666 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.065721 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.065976 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.066518 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.067330 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.067520 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.067597 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.068358 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:23.068359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:23.068360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:23.068367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:23.069485 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.069868 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.069869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.069869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.069922 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.070197 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.070198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.070198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.070297 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:23.070383 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.070534 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.070620 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:23.093397 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45985
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:23.173413 2477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:23.173646 2477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:23.173682 2477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:23.174818 2477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:23.174855 2477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:23.174875 2477 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:23.174943 2477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:23.176319 2477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:45985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:23.176570 2477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:23.176808 2477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:23.181535 2482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:23.181524 2485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:23.181540 2477 server_base.cc:1034] running on GCE node
W20260620 23:08:23.181521 2483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:23.181974 2477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:23.182296 2477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:23.183439 2477 hybrid_clock.cc:648] HybridClock initialized: now 1781996903183435 us; error 37 us; skew 500 ppm
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:23.185853 2477 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:23.186269 2477 webserver.cc:460] Webserver started at http://127.1.32.66:46273/ using document root <none> and password file <none>
I20260620 23:08:23.186499 2477 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:23.186565 2477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:23.186712 2477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:23.187814 2477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "e432f20b7ad940ce94bb55e9ec6a92b1"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "ee1c29c9d421cbafdb4eb3db1b69535f"
server_key_iv: "1a3e1b9acea628897b770ee83ac664b5"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.188172 2477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "e432f20b7ad940ce94bb55e9ec6a92b1"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "ee1c29c9d421cbafdb4eb3db1b69535f"
server_key_iv: "1a3e1b9acea628897b770ee83ac664b5"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.189456 2477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:23.190210 2492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.190418 2477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:23.190516 2477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "e432f20b7ad940ce94bb55e9ec6a92b1"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "ee1c29c9d421cbafdb4eb3db1b69535f"
server_key_iv: "1a3e1b9acea628897b770ee83ac664b5"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.190596 2477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:23.219519 2477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:23.221585 2477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:23.221761 2477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:23.222420 2477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:23.222491 2477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.222537 2477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:23.222569 2477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:23.230430 2477 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:46783
I20260620 23:08:23.230506 2607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:46783 every 8 connection(s)
I20260620 23:08:23.230908 2477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:23.232860 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.222917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:48055 (local address 127.1.32.126:45985)
0620 23:08:23.222968 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.222970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.223262 (+ 292us) server_negotiation.cc:407] Connection header received
0620 23:08:23.224000 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.224001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.224014 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.224076 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.225014 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.225857 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.227507 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.227751 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.227836 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.230081 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:23.230082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:23.230082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:23.230090 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:23.231725 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.232088 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.232090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.232090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.232147 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.232375 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.232375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.232376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.232475 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:23.232525 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.232740 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.232799 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:08:23.233001 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.222881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45985 (local address 127.1.32.66:48055)
0620 23:08:23.223099 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.223164 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.223879 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.224168 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.224173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.224590 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:23.224899 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.224908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.225973 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.225974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.227415 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.227419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.227856 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.227857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.228030 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.228563 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:23.228573 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:23.229960 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:23.231796 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.231800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.231802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.232003 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.232226 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.232226 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.232227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.232313 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.232528 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:23.232531 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:23.232634 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.232722 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.232828 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260620 23:08:23.233428 2608 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45985
I20260620 23:08:23.233505 2608 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:23.233737 2608 heartbeater.cc:511] Master 127.1.32.126:45985 requested a full tablet report, sending...
I20260620 23:08:23.234143 2264 ts_manager.cc:194] Registered new tserver with Master: e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783)
I20260620 23:08:23.234660 2264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:48055
I20260620 23:08:23.238332 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 2477
I20260620 23:08:23.238435 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:23.238628 1153 external_mini_cluster.cc:1428] Setting key c43603e3fe0be185f16499f131437975
I20260620 23:08:23.240154 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.234419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45985 (local address 127.1.32.66:53681)
0620 23:08:23.234547 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.234557 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.234609 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.234943 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.234944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.235139 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:23.235214 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.235216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.235819 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.235820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.236299 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.236300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.236618 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.236620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.236682 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.237120 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:23.237124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:23.237392 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:23.239061 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.239062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.239063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.239262 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.239451 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.239451 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.239452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.239512 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.239877 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:23.239878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:23.239943 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.240016 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.240088 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260620 23:08:23.240304 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.234486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:53681 (local address 127.1.32.126:45985)
0620 23:08:23.234810 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.234812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.234820 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:23.234841 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.234841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.234851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.234937 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.235288 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.235733 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.236366 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.236533 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.236584 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.237460 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:23.237461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:23.237461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:23.237465 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:23.238973 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.239328 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.239329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.239329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.239382 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.239583 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.239583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.239584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.239804 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:23.239883 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.240061 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.240147 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:23.261274 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45985
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:23.339771 2617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:23.340032 2617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:23.340070 2617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:23.341356 2617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:23.341424 2617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:23.341461 2617 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:23.341522 2617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:23.342924 2617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:45985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:23.343173 2617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:23.343394 2617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:23.348240 2622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:23.348318 2623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:23.348373 2617 server_base.cc:1034] running on GCE node
W20260620 23:08:23.348263 2625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:23.348626 2617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:23.348834 2617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:23.349979 2617 hybrid_clock.cc:648] HybridClock initialized: now 1781996903349956 us; error 40 us; skew 500 ppm
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:23.352486 2617 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:23.352931 2617 webserver.cc:460] Webserver started at http://127.1.32.67:46043/ using document root <none> and password file <none>
I20260620 23:08:23.353145 2617 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:23.353224 2617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:23.353372 2617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:23.354382 2617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "1dd7844373ee4ccfb9075604d269714b"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "56f7cdade6a5a2fd31e25086d2567906"
server_key_iv: "fad27cc67d7552dea622b36fb38633d8"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.354873 2617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "1dd7844373ee4ccfb9075604d269714b"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "56f7cdade6a5a2fd31e25086d2567906"
server_key_iv: "fad27cc67d7552dea622b36fb38633d8"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.356221 2617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:23.356904 2632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.357072 2617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.357163 2617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "1dd7844373ee4ccfb9075604d269714b"
format_stamp: "Formatted at 2026-06-20 23:08:23 on dist-test-slave-fzjm"
server_key: "56f7cdade6a5a2fd31e25086d2567906"
server_key_iv: "fad27cc67d7552dea622b36fb38633d8"
server_key_version: "encryptionkey@0"
I20260620 23:08:23.357249 2617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:23.385036 2617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:23.387118 2617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:23.387276 2617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:23.387758 2617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:23.387817 2617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.387874 2617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:23.387903 2617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.395116 2617 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:40519
I20260620 23:08:23.395174 2747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:40519 every 8 connection(s)
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:23.395507 2617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:08:23.398459 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.388319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:56083 (local address 127.1.32.126:45985)
0620 23:08:23.388390 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.388393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.388628 (+ 235us) server_negotiation.cc:407] Connection header received
0620 23:08:23.389368 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.389370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.389388 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.389425 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.390404 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.391204 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.392840 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.393150 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.393225 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.395948 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:23.395950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:23.395950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:23.395957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:23.397354 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.397705 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.397706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.397706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.397763 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.397979 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.397980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.397980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.398072 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:23.398119 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.398344 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.398402 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:08:23.398604 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.388300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45985 (local address 127.1.32.67:56083)
0620 23:08:23.388487 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.388535 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.389223 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.389618 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.389622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.390010 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:23.390304 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.390312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.391292 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.391293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.392748 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.392751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.393248 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.393249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.393418 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.393993 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:23.394003 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:23.395842 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:23.397426 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.397430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.397432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.397621 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.397821 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.397821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.397822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.397918 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.398124 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:23.398128 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:23.398244 (+ 116us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.398319 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.398427 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260620 23:08:23.399070 2748 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45985
I20260620 23:08:23.399138 2748 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:23.399317 2748 heartbeater.cc:511] Master 127.1.32.126:45985 requested a full tablet report, sending...
I20260620 23:08:23.399705 2265 ts_manager.cc:194] Registered new tserver with Master: 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519)
I20260620 23:08:23.400044 2265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:56083
I20260620 23:08:23.404806 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.400122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:32965 (local address 127.1.32.126:45985)
0620 23:08:23.400158 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.400160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.400271 (+ 111us) server_negotiation.cc:407] Connection header received
0620 23:08:23.400360 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.400360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.400371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.400420 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.400688 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.401172 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.401805 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.401975 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.402029 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.402842 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:23.402843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:23.402843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:23.402848 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:23.403981 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.404254 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.404270 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.404271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.404316 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.404474 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.404474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.404475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.404554 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:23.404597 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.404703 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.404753 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:08:23.404807 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.400075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45985 (local address 127.1.32.67:32965)
0620 23:08:23.400214 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.400224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.400284 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.400430 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.400431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.400568 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:23.400615 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.400616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.401249 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.401250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.401731 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.401732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.402038 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.402039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.402087 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.402496 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:23.402501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:23.402776 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:23.404046 (+ 1270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.404047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.404048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.404195 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.404370 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:23.404370 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:23.404370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:23.404426 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:23.404600 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:23.404600 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:23.404631 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.404678 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.404748 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260620 23:08:23.405496 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 2617
I20260620 23:08:23.405592 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:23.405771 1153 external_mini_cluster.cc:1428] Setting key 7cdde787cc8f88d71bc87aacf87c532c
I20260620 23:08:23.406530 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:23.441116 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.432638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55880 (local address 127.1.32.126:45985)
0620 23:08:23.432723 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.432726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.432738 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:08:23.432769 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.432770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.432784 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.432863 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.433953 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.434864 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.436004 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.436232 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.436283 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.438368 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:23.438369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:23.438369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:23.438376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:23.439921 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.440257 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.440258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.440259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.440313 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:23.440499 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:23.440500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:23.440501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:23.440616 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:23.440742 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.440919 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.441000 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 23:08:23.442873 2265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:23.444490 2265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:23.451012 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.446979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:46783 (local address 127.0.0.1:33110)
0620 23:08:23.447250 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.447259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.447307 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.447581 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.447582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.447583 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:23.447627 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.447628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.448582 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.448583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.449997 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.450000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.450676 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.450763 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.450776 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.450844 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.450918 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":59,"thread_start_us":75,"threads_started":1}
I20260620 23:08:23.451108 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.447152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:35879 (local address 127.0.0.1:44200)
0620 23:08:23.447647 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.447656 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.447702 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.447943 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.447944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.447946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:23.447982 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.447984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.449051 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.449052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.449955 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.449957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.450832 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.450913 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.450924 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.450981 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.451011 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":66,"threads_started":1}
I20260620 23:08:23.451136 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.447289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.1.32.65:35879)
0620 23:08:23.447448 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.447453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.447769 (+ 316us) server_negotiation.cc:407] Connection header received
0620 23:08:23.447809 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.447811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.447825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.447886 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:23.448045 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.448974 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450032 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.450748 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450833 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.450933 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.450994 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.451029 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260620 23:08:23.451427 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.446841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.1.32.67:40519)
0620 23:08:23.447129 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.447150 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.447160 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:23.447200 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.447202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.447219 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.447298 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:23.447465 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.448594 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450119 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.450780 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450837 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.450923 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.451114 (+ 191us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.451363 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":177,"threads_started":1}
I20260620 23:08:23.451664 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.446727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:40519 (local address 127.0.0.1:46312)
0620 23:08:23.446849 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.446913 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.446984 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.447319 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.447323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.447325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:23.447403 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.447405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.448670 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.448670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.449998 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.450000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.450959 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.451023 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.451027 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.451575 (+ 548us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.451620 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":58,"thread_start_us":53,"threads_started":1}
I20260620 23:08:23.451669 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.447079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33110 (local address 127.1.32.66:46783)
0620 23:08:23.447422 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.447426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.447435 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:23.447469 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.447471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.447485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.447566 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:23.447693 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.448492 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450092 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.450575 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.450677 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.450739 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.451538 (+ 799us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.451582 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":72,"threads_started":1}
I20260620 23:08:23.451993 2402 tablet_service.cc:1467] Processing CreateTablet for tablet 07d408f57fdd4041a72a9a36211f1d77 (DEFAULT_TABLE table=test-table [id=83b0a2c81537412d9276d849dccb0b9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:23.452380 2402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d408f57fdd4041a72a9a36211f1d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:23.452353 2667 tablet_service.cc:1467] Processing CreateTablet for tablet 07d408f57fdd4041a72a9a36211f1d77 (DEFAULT_TABLE table=test-table [id=83b0a2c81537412d9276d849dccb0b9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:23.452677 2667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d408f57fdd4041a72a9a36211f1d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:23.455857 2771 tablet_bootstrap.cc:492] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Bootstrap starting.
I20260620 23:08:23.456096 2772 tablet_bootstrap.cc:492] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Bootstrap starting.
I20260620 23:08:23.456595 2542 tablet_service.cc:1467] Processing CreateTablet for tablet 07d408f57fdd4041a72a9a36211f1d77 (DEFAULT_TABLE table=test-table [id=83b0a2c81537412d9276d849dccb0b9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:23.456805 2772 tablet_bootstrap.cc:654] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:23.456923 2542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d408f57fdd4041a72a9a36211f1d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:23.456931 2771 tablet_bootstrap.cc:654] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:23.457147 2772 log.cc:824] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:23.457269 2771 log.cc:824] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:23.457976 2772 tablet_bootstrap.cc:492] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: No bootstrap required, opened a new log
I20260620 23:08:23.457985 2771 tablet_bootstrap.cc:492] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: No bootstrap required, opened a new log
I20260620 23:08:23.458050 2771 ts_tablet_manager.cc:1397] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:08:23.458050 2772 ts_tablet_manager.cc:1397] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:23.460081 2772 raft_consensus.cc:348] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.460134 2771 raft_consensus.cc:348] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.460253 2772 raft_consensus.cc:374] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:23.460253 2771 raft_consensus.cc:374] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:23.460286 2771 raft_consensus.cc:729] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd7844373ee4ccfb9075604d269714b, State: Initialized, Role: FOLLOWER
I20260620 23:08:23.460286 2772 raft_consensus.cc:729] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f918fa8bd8b429186675e60017c2b0f, State: Initialized, Role: FOLLOWER
I20260620 23:08:23.460160 2775 tablet_bootstrap.cc:492] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Bootstrap starting.
I20260620 23:08:23.460417 2771 consensus_queue.cc:260] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.460407 2772 consensus_queue.cc:260] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.460671 2772 ts_tablet_manager.cc:1428] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:23.460671 2771 ts_tablet_manager.cc:1428] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:23.460804 2468 heartbeater.cc:503] Master 127.1.32.126:45985 was elected leader, sending a full tablet report...
I20260620 23:08:23.460811 2748 heartbeater.cc:503] Master 127.1.32.126:45985 was elected leader, sending a full tablet report...
I20260620 23:08:23.461010 2775 tablet_bootstrap.cc:654] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:23.461311 2775 log.cc:824] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:23.462160 2775 tablet_bootstrap.cc:492] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: No bootstrap required, opened a new log
I20260620 23:08:23.462222 2775 ts_tablet_manager.cc:1397] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:23.463795 2775 raft_consensus.cc:348] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.463933 2775 raft_consensus.cc:374] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:23.463955 2775 raft_consensus.cc:729] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e432f20b7ad940ce94bb55e9ec6a92b1, State: Initialized, Role: FOLLOWER
I20260620 23:08:23.464088 2775 consensus_queue.cc:260] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.464329 2775 ts_tablet_manager.cc:1428] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:08:23.464365 2608 heartbeater.cc:503] Master 127.1.32.126:45985 was elected leader, sending a full tablet report...
W20260620 23:08:23.481536 2609 tablet.cc:2369] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:08:23.562106 2469 tablet.cc:2369] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:08:23.646116 2749 tablet.cc:2369] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:23.666769 2777 raft_consensus.cc:482] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:23.666875 2777 raft_consensus.cc:504] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.667253 2777 leader_election.cc:288] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519), e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783)
I20260620 23:08:23.670184 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.667743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:48221 (local address 127.1.32.66:46783)
0620 23:08:23.667787 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.667789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.667797 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:23.667865 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.667865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.667877 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.667944 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:23.668109 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.668550 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.669514 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.669855 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.669904 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.669911 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.670073 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.670107 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:08:23.670210 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.667436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:40519 (local address 127.1.32.65:38929)
0620 23:08:23.667576 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.667590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.667712 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.667867 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.667868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.667870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:23.667955 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.667957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.668576 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.668577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.669419 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.669421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.669961 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.669963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.670016 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.670019 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.670083 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.670116 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":28,"thread_start_us":72,"threads_started":1}
I20260620 23:08:23.670207 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.667539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:38929 (local address 127.1.32.67:40519)
0620 23:08:23.667600 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.667602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.667692 (+ 90us) server_negotiation.cc:407] Connection header received
0620 23:08:23.667784 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.667785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.667798 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.667856 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:23.668022 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.668486 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.669512 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.669860 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.669946 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.669954 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.670107 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.670143 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:08:23.670217 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.667621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:46783 (local address 127.1.32.65:48221)
0620 23:08:23.667729 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.667738 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.667788 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.667968 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.667969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.667971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:23.668045 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.668047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.668633 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.668634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.669422 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.669424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.669934 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.669935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.670001 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.670005 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:23.670051 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 23:08:23.670107 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":40,"thread_start_us":42,"threads_started":1}
I20260620 23:08:23.670498 2702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d408f57fdd4041a72a9a36211f1d77" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd7844373ee4ccfb9075604d269714b" is_pre_election: true
I20260620 23:08:23.670639 2702 raft_consensus.cc:2454] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 0.
I20260620 23:08:23.670935 2356 leader_election.cc:302] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dd7844373ee4ccfb9075604d269714b, 7f918fa8bd8b429186675e60017c2b0f; no voters:
I20260620 23:08:23.671027 2562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d408f57fdd4041a72a9a36211f1d77" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" is_pre_election: true
I20260620 23:08:23.671059 2777 raft_consensus.cc:2781] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:23.671108 2777 raft_consensus.cc:482] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:23.671129 2777 raft_consensus.cc:3037] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:23.671157 2562 raft_consensus.cc:2454] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 0.
I20260620 23:08:23.671869 2777 raft_consensus.cc:504] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.671989 2777 leader_election.cc:288] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 election: Requested vote from peers 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519), e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783)
I20260620 23:08:23.672145 2562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d408f57fdd4041a72a9a36211f1d77" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1"
I20260620 23:08:23.672163 2702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d408f57fdd4041a72a9a36211f1d77" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd7844373ee4ccfb9075604d269714b"
I20260620 23:08:23.672231 2562 raft_consensus.cc:3037] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:23.672250 2702 raft_consensus.cc:3037] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:23.673022 2702 raft_consensus.cc:2454] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 1.
I20260620 23:08:23.673029 2562 raft_consensus.cc:2454] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 1.
I20260620 23:08:23.673210 2356 leader_election.cc:302] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f918fa8bd8b429186675e60017c2b0f, e432f20b7ad940ce94bb55e9ec6a92b1; no voters:
I20260620 23:08:23.673323 2777 raft_consensus.cc:2781] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:23.673460 2777 raft_consensus.cc:686] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 1 LEADER]: Becoming Leader. State: Replica: 7f918fa8bd8b429186675e60017c2b0f, State: Running, Role: LEADER
I20260620 23:08:23.673552 2777 consensus_queue.cc:237] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } }
I20260620 23:08:23.674382 2264 catalog_manager.cc:5495] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f918fa8bd8b429186675e60017c2b0f (127.1.32.65). New cstate: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } health_report { overall_health: HEALTHY } } }
I20260620 23:08:23.732836 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.730446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.1.32.65:35879)
0620 23:08:23.730506 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.730508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.730523 (+ 15us) server_negotiation.cc:407] Connection header received
0620 23:08:23.730558 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.730559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.730578 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.730640 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:23.730801 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.731335 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.732122 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.732307 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.732354 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.732530 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:23.732619 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:23.732675 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:23.732724 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 23:08:23.732766 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:08:23.734717 2562 raft_consensus.cc:1261] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Refusing update from remote peer 7f918fa8bd8b429186675e60017c2b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:23.734728 2702 raft_consensus.cc:1261] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Refusing update from remote peer 7f918fa8bd8b429186675e60017c2b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:23.735095 2777 consensus_queue.cc:1035] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:23.735245 2782 consensus_queue.cc:1035] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:23.737712 2787 mvcc.cc:205] Tried to move back new op lower bound from 7299059317694312448 to 7299059317447512064. Current Snapshot: MvccSnapshot[applied={T|T < 7299059317694312448}]
I20260620 23:08:23.737864 2786 mvcc.cc:205] Tried to move back new op lower bound from 7299059317694312448 to 7299059317447512064. Current Snapshot: MvccSnapshot[applied={T|T < 7299059317694312448}]
I20260620 23:08:23.737995 2785 mvcc.cc:205] Tried to move back new op lower bound from 7299059317694312448 to 7299059317447512064. Current Snapshot: MvccSnapshot[applied={T|T < 7299059317694312448}]
I20260620 23:08:23.741271 2264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:23.741370 2264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:23.742084 2264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c: Sending DeleteTablet for 3 replicas of tablet 07d408f57fdd4041a72a9a36211f1d77
I20260620 23:08:23.742384 2667 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d408f57fdd4041a72a9a36211f1d77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:23 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:46312
I20260620 23:08:23.742478 2402 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d408f57fdd4041a72a9a36211f1d77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:23 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:44200
I20260620 23:08:23.742594 2542 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d408f57fdd4041a72a9a36211f1d77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:23 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:33110
I20260620 23:08:23.742871 2795 tablet_replica.cc:330] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: stopping tablet replica
I20260620 23:08:23.742830 2264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:23.742980 2264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:23.742993 2795 raft_consensus.cc:2229] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:23.743083 2795 raft_consensus.cc:2258] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:23.743222 2797 tablet_replica.cc:330] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: stopping tablet replica
I20260620 23:08:23.743352 2797 raft_consensus.cc:2229] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:23.743490 2797 raft_consensus.cc:2258] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:23.743510 2795 ts_tablet_manager.cc:1905] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:23.743752 2796 tablet_replica.cc:330] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: stopping tablet replica
I20260620 23:08:23.743878 2796 raft_consensus.cc:2229] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:23.744066 2796 raft_consensus.cc:2258] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:23.744419 2402 tablet_service.cc:1467] Processing CreateTablet for tablet af1473ab1ab74b378eca3951cc648276 (DEFAULT_TABLE table=test-table [id=57b691c146154bc495eeae8997470bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:23.744529 2542 tablet_service.cc:1467] Processing CreateTablet for tablet af1473ab1ab74b378eca3951cc648276 (DEFAULT_TABLE table=test-table [id=57b691c146154bc495eeae8997470bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:23.744551 2402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1473ab1ab74b378eca3951cc648276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:23.744616 2542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1473ab1ab74b378eca3951cc648276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:23.744611 2796 ts_tablet_manager.cc:1905] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:23.745198 2795 ts_tablet_manager.cc:1918] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:23.745261 2795 log.cc:1196] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/07d408f57fdd4041a72a9a36211f1d77
I20260620 23:08:23.745522 2795 ts_tablet_manager.cc:1939] T 07d408f57fdd4041a72a9a36211f1d77 P 1dd7844373ee4ccfb9075604d269714b: Deleting consensus metadata
I20260620 23:08:23.745887 2772 tablet_bootstrap.cc:492] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: Bootstrap starting.
I20260620 23:08:23.745903 2775 tablet_bootstrap.cc:492] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: Bootstrap starting.
I20260620 23:08:23.746011 2667 tablet_service.cc:1467] Processing CreateTablet for tablet af1473ab1ab74b378eca3951cc648276 (DEFAULT_TABLE table=test-table [id=57b691c146154bc495eeae8997470bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:23.746140 2667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1473ab1ab74b378eca3951cc648276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:23.746215 2796 ts_tablet_manager.cc:1918] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:23.746276 2796 log.cc:1196] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/07d408f57fdd4041a72a9a36211f1d77
I20260620 23:08:23.746320 2775 tablet_bootstrap.cc:654] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:23.746335 2772 tablet_bootstrap.cc:654] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:23.746609 2796 ts_tablet_manager.cc:1939] T 07d408f57fdd4041a72a9a36211f1d77 P 7f918fa8bd8b429186675e60017c2b0f: Deleting consensus metadata
I20260620 23:08:23.746670 2252 catalog_manager.cc:4841] TS 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519): tablet 07d408f57fdd4041a72a9a36211f1d77 (table test-table [id=83b0a2c81537412d9276d849dccb0b9b]) successfully deleted
I20260620 23:08:23.747138 2772 tablet_bootstrap.cc:492] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: No bootstrap required, opened a new log
I20260620 23:08:23.747184 2772 ts_tablet_manager.cc:1397] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:23.747318 2772 raft_consensus.cc:348] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.747402 2772 raft_consensus.cc:374] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:23.747426 2772 raft_consensus.cc:729] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f918fa8bd8b429186675e60017c2b0f, State: Initialized, Role: FOLLOWER
I20260620 23:08:23.747465 2771 tablet_bootstrap.cc:492] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: Bootstrap starting.
I20260620 23:08:23.747467 2772 consensus_queue.cc:260] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.747572 2772 ts_tablet_manager.cc:1428] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:23.747850 2771 tablet_bootstrap.cc:654] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:23.747960 2797 ts_tablet_manager.cc:1905] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:23.748451 2771 tablet_bootstrap.cc:492] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: No bootstrap required, opened a new log
I20260620 23:08:23.748504 2771 ts_tablet_manager.cc:1397] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:23.748628 2771 raft_consensus.cc:348] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.748696 2771 raft_consensus.cc:374] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:23.748718 2771 raft_consensus.cc:729] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd7844373ee4ccfb9075604d269714b, State: Initialized, Role: FOLLOWER
I20260620 23:08:23.748756 2771 consensus_queue.cc:260] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.748854 2771 ts_tablet_manager.cc:1428] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:23.749580 2797 ts_tablet_manager.cc:1918] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:23.749646 2797 log.cc:1196] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/07d408f57fdd4041a72a9a36211f1d77
I20260620 23:08:23.749869 2797 ts_tablet_manager.cc:1939] T 07d408f57fdd4041a72a9a36211f1d77 P e432f20b7ad940ce94bb55e9ec6a92b1: Deleting consensus metadata
I20260620 23:08:23.750242 2252 catalog_manager.cc:4841] TS e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783): tablet 07d408f57fdd4041a72a9a36211f1d77 (table test-table [id=83b0a2c81537412d9276d849dccb0b9b]) successfully deleted
I20260620 23:08:23.751224 2252 catalog_manager.cc:4841] TS 7f918fa8bd8b429186675e60017c2b0f (127.1.32.65:35879): tablet 07d408f57fdd4041a72a9a36211f1d77 (table test-table [id=83b0a2c81537412d9276d849dccb0b9b]) successfully deleted
I20260620 23:08:23.751276 2775 tablet_bootstrap.cc:492] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: No bootstrap required, opened a new log
I20260620 23:08:23.751313 2775 ts_tablet_manager.cc:1397] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260620 23:08:23.751592 2775 raft_consensus.cc:348] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.751647 2775 raft_consensus.cc:374] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:23.751660 2775 raft_consensus.cc:729] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e432f20b7ad940ce94bb55e9ec6a92b1, State: Initialized, Role: FOLLOWER
I20260620 23:08:23.751715 2775 consensus_queue.cc:260] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.751827 2775 ts_tablet_manager.cc:1428] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:23.945201 2790 raft_consensus.cc:482] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:23.945299 2790 raft_consensus.cc:504] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.945520 2790 leader_election.cc:288] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783), 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519)
I20260620 23:08:23.945792 2562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1473ab1ab74b378eca3951cc648276" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" is_pre_election: true
I20260620 23:08:23.945816 2702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1473ab1ab74b378eca3951cc648276" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd7844373ee4ccfb9075604d269714b" is_pre_election: true
I20260620 23:08:23.945899 2702 raft_consensus.cc:2454] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 0.
I20260620 23:08:23.945902 2562 raft_consensus.cc:2454] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 0.
I20260620 23:08:23.946066 2356 leader_election.cc:302] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dd7844373ee4ccfb9075604d269714b, 7f918fa8bd8b429186675e60017c2b0f; no voters:
I20260620 23:08:23.946188 2790 raft_consensus.cc:2781] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:23.946247 2790 raft_consensus.cc:482] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:23.946278 2790 raft_consensus.cc:3037] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:23.947077 2790 raft_consensus.cc:504] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.947211 2790 leader_election.cc:288] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 election: Requested vote from peers e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783), 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519)
I20260620 23:08:23.947363 2562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1473ab1ab74b378eca3951cc648276" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1"
I20260620 23:08:23.947403 2702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1473ab1ab74b378eca3951cc648276" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd7844373ee4ccfb9075604d269714b"
I20260620 23:08:23.947444 2562 raft_consensus.cc:3037] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:23.947467 2702 raft_consensus.cc:3037] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:23.948182 2562 raft_consensus.cc:2454] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 1.
I20260620 23:08:23.948190 2702 raft_consensus.cc:2454] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 1.
I20260620 23:08:23.948380 2356 leader_election.cc:302] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f918fa8bd8b429186675e60017c2b0f, e432f20b7ad940ce94bb55e9ec6a92b1; no voters:
I20260620 23:08:23.948479 2790 raft_consensus.cc:2781] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:23.948534 2790 raft_consensus.cc:686] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 1 LEADER]: Becoming Leader. State: Replica: 7f918fa8bd8b429186675e60017c2b0f, State: Running, Role: LEADER
I20260620 23:08:23.948583 2790 consensus_queue.cc:237] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:23.949163 2264 catalog_manager.cc:5495] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f918fa8bd8b429186675e60017c2b0f (127.1.32.65). New cstate: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } health_report { overall_health: UNKNOWN } } }
Jun 20 23:08:23 dist-test-slave-fzjm krb5kdc[2220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996902, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:24.003441 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.991483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45985 (local address 127.0.0.1:55896)
0620 23:08:23.991690 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:23.991707 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:23.991814 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:23.991998 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:23.991998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:23.992634 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:23.992683 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.992686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.993404 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.993405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.993995 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:23.993997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.994362 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.994363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:23.994413 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:23.999381 (+ 4968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:23.999386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:24.000705 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:24.002246 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:24.002254 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:24.002256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:24.002466 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:24.002679 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:24.002680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:24.002680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:24.002752 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:24.003054 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:24.003061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:24.003109 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 23:08:24.003193 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:08:24.003322 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260620 23:08:24.003449 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:23.991544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.1.32.126:45985)
0620 23:08:23.991670 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:23.991672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:23.991794 (+ 122us) server_negotiation.cc:407] Connection header received
0620 23:08:23.991896 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:23.991897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:23.991908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:23.991981 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:23.992792 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.993318 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.994091 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:23.994283 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:23.994363 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:24.000806 (+ 6443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:24.000807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:24.000808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:24.000814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:24.002110 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:24.002557 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:24.002558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:24.002558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:24.002616 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:24.002817 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:24.002818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:24.002818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:24.002992 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:24.003076 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:24.003234 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 23:08:24.003373 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260620 23:08:24.005344 2264 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:55896:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 23:08:24.005589 2264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:24.006973 2402 tablet_service.cc:1467] Processing CreateTablet for tablet 476a95dc3b634798b711d70931bb09e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e25efd215694ba9b1257c1db1e3dc89]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:24.007066 2542 tablet_service.cc:1467] Processing CreateTablet for tablet 476a95dc3b634798b711d70931bb09e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e25efd215694ba9b1257c1db1e3dc89]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:24.007133 2402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476a95dc3b634798b711d70931bb09e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:24.007225 2542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476a95dc3b634798b711d70931bb09e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:24.008301 2772 tablet_bootstrap.cc:492] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: Bootstrap starting.
I20260620 23:08:24.008390 2667 tablet_service.cc:1467] Processing CreateTablet for tablet 476a95dc3b634798b711d70931bb09e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e25efd215694ba9b1257c1db1e3dc89]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:24.008514 2667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476a95dc3b634798b711d70931bb09e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:24.008630 2772 tablet_bootstrap.cc:654] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:24.009238 2772 tablet_bootstrap.cc:492] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: No bootstrap required, opened a new log
I20260620 23:08:24.009281 2772 ts_tablet_manager.cc:1397] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:24.009405 2772 raft_consensus.cc:348] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.009470 2772 raft_consensus.cc:374] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:24.009485 2772 raft_consensus.cc:729] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f918fa8bd8b429186675e60017c2b0f, State: Initialized, Role: FOLLOWER
I20260620 23:08:24.009531 2772 consensus_queue.cc:260] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.009626 2772 ts_tablet_manager.cc:1428] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:24.009645 2790 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.009779 2790 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:24.010115 2771 tablet_bootstrap.cc:492] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: Bootstrap starting.
I20260620 23:08:24.010167 2775 tablet_bootstrap.cc:492] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: Bootstrap starting.
I20260620 23:08:24.010483 2771 tablet_bootstrap.cc:654] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:24.010504 2775 tablet_bootstrap.cc:654] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:24.011149 2775 tablet_bootstrap.cc:492] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: No bootstrap required, opened a new log
I20260620 23:08:24.011229 2775 ts_tablet_manager.cc:1397] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:24.011250 2771 tablet_bootstrap.cc:492] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: No bootstrap required, opened a new log
I20260620 23:08:24.011303 2771 ts_tablet_manager.cc:1397] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:24.011377 2775 raft_consensus.cc:348] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.011452 2775 raft_consensus.cc:374] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:24.011473 2775 raft_consensus.cc:729] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e432f20b7ad940ce94bb55e9ec6a92b1, State: Initialized, Role: FOLLOWER
I20260620 23:08:24.011441 2771 raft_consensus.cc:348] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.011505 2771 raft_consensus.cc:374] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:24.011530 2771 raft_consensus.cc:729] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd7844373ee4ccfb9075604d269714b, State: Initialized, Role: FOLLOWER
I20260620 23:08:24.011514 2775 consensus_queue.cc:260] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.011579 2771 consensus_queue.cc:260] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.011606 2775 ts_tablet_manager.cc:1428] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:24.011668 2771 ts_tablet_manager.cc:1428] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:24.011687 2779 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.011768 2779 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:24.011793 2776 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.011868 2776 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:24.058445 2790 raft_consensus.cc:482] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:24.058527 2790 raft_consensus.cc:504] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.058709 2790 leader_election.cc:288] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783), 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519)
I20260620 23:08:24.058933 2562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "476a95dc3b634798b711d70931bb09e3" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" is_pre_election: true
I20260620 23:08:24.058965 2702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "476a95dc3b634798b711d70931bb09e3" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd7844373ee4ccfb9075604d269714b" is_pre_election: true
I20260620 23:08:24.059029 2562 raft_consensus.cc:2454] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 0.
I20260620 23:08:24.059041 2702 raft_consensus.cc:2454] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 0.
I20260620 23:08:24.059168 2356 leader_election.cc:302] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f918fa8bd8b429186675e60017c2b0f, e432f20b7ad940ce94bb55e9ec6a92b1; no voters:
I20260620 23:08:24.059278 2790 raft_consensus.cc:2781] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:24.059324 2790 raft_consensus.cc:482] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:24.059345 2790 raft_consensus.cc:3037] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:24.059973 2790 raft_consensus.cc:504] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.060096 2790 leader_election.cc:288] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 election: Requested vote from peers e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783), 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519)
I20260620 23:08:24.060300 2562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "476a95dc3b634798b711d70931bb09e3" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1"
I20260620 23:08:24.060381 2562 raft_consensus.cc:3037] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:24.060346 2702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "476a95dc3b634798b711d70931bb09e3" candidate_uuid: "7f918fa8bd8b429186675e60017c2b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd7844373ee4ccfb9075604d269714b"
I20260620 23:08:24.060437 2702 raft_consensus.cc:3037] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:24.061141 2562 raft_consensus.cc:2454] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 1.
I20260620 23:08:24.061146 2702 raft_consensus.cc:2454] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f918fa8bd8b429186675e60017c2b0f in term 1.
I20260620 23:08:24.061345 2356 leader_election.cc:302] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dd7844373ee4ccfb9075604d269714b, 7f918fa8bd8b429186675e60017c2b0f; no voters:
I20260620 23:08:24.061465 2790 raft_consensus.cc:2781] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:24.061523 2790 raft_consensus.cc:686] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [term 1 LEADER]: Becoming Leader. State: Replica: 7f918fa8bd8b429186675e60017c2b0f, State: Running, Role: LEADER
I20260620 23:08:24.061576 2790 consensus_queue.cc:237] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } }
I20260620 23:08:24.061925 2790 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: TxnStatusTablet state changed. Reason: New leader 7f918fa8bd8b429186675e60017c2b0f. Latest consensus state: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.061995 2790 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:24.062206 2813 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 23:08:24.062325 2813 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 23:08:24.063027 2265 catalog_manager.cc:5495] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f918fa8bd8b429186675e60017c2b0f (127.1.32.65). New cstate: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:24.070456 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:24.067916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.1.32.65:35879)
0620 23:08:24.067956 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:24.067958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:24.068022 (+ 64us) server_negotiation.cc:407] Connection header received
0620 23:08:24.068109 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:24.068110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:24.068118 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:24.068162 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:24.068313 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.068779 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.069857 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.070026 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.070073 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:24.070253 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:24.070276 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:24.070313 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:24.070393 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:08:24.070411 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:24.070518 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:24.067876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:35879 (local address 127.0.0.1:44222)
0620 23:08:24.067960 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:24.067974 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:24.068040 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:24.068197 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:24.068198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:24.068199 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:24.068249 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:24.068252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.068859 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.068860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:24.069771 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:24.069773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.070093 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.070094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:24.070153 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:24.070169 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:24.070331 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:24.070338 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:08:24.070392 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:08:24.070443 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 23:08:24.071347 2702 raft_consensus.cc:1261] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Refusing update from remote peer 7f918fa8bd8b429186675e60017c2b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:24.071385 2562 raft_consensus.cc:1261] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Refusing update from remote peer 7f918fa8bd8b429186675e60017c2b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:24.071561 2777 consensus_queue.cc:1035] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:24.071658 2790 consensus_queue.cc:1035] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:24.072144 2776 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: TxnStatusTablet state changed. Reason: New leader 7f918fa8bd8b429186675e60017c2b0f. Latest consensus state: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.072239 2776 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:24.072265 2779 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: TxnStatusTablet state changed. Reason: New leader 7f918fa8bd8b429186675e60017c2b0f. Latest consensus state: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.072353 2779 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:24.073002 2782 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.073094 2782 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:24.073168 2782 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.073161 2776 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.073240 2782 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P 7f918fa8bd8b429186675e60017c2b0f: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:24.073247 2776 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P 1dd7844373ee4ccfb9075604d269714b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:24.074165 2779 tablet_replica.cc:439] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f918fa8bd8b429186675e60017c2b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f918fa8bd8b429186675e60017c2b0f" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35879 } } peers { permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 } } peers { permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 } } }
I20260620 23:08:24.074277 2779 tablet_replica.cc:442] T 476a95dc3b634798b711d70931bb09e3 P e432f20b7ad940ce94bb55e9ec6a92b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:24.078780 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:24.076623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:56477 (local address 127.1.32.65:35879)
0620 23:08:24.076662 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:24.076664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:24.076760 (+ 96us) server_negotiation.cc:407] Connection header received
0620 23:08:24.076820 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:24.076820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:24.076829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:24.076883 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:24.076979 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.077420 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.078125 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.078313 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.078394 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:24.078535 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:24.078573 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:24.078631 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:24.078688 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 23:08:24.078708 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:08:24.078799 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:24.076561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:35879 (local address 127.1.32.65:56477)
0620 23:08:24.076685 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:24.076694 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:24.076775 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:24.076885 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:24.076886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:24.076887 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:24.076923 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:24.076924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.077499 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.077500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:24.078045 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:24.078047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.078397 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.078398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:24.078450 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:24.078461 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:24.078632 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:24.078633 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:08:24.078685 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:08:24.078725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260620 23:08:24.080813 2702 raft_consensus.cc:1261] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Refusing update from remote peer 7f918fa8bd8b429186675e60017c2b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:24.081039 2790 consensus_queue.cc:1035] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dd7844373ee4ccfb9075604d269714b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:24.081076 2562 raft_consensus.cc:1261] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Refusing update from remote peer 7f918fa8bd8b429186675e60017c2b0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:24.081887 2790 consensus_queue.cc:1035] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e432f20b7ad940ce94bb55e9ec6a92b1" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:24.094931 2264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:24.095014 2264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:24.095903 2264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3eda54aa7e644bab939117dd70e89c7c: Sending DeleteTablet for 3 replicas of tablet af1473ab1ab74b378eca3951cc648276
I20260620 23:08:24.096207 2402 tablet_service.cc:1514] Processing DeleteTablet for tablet af1473ab1ab74b378eca3951cc648276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:24 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:44200
I20260620 23:08:24.096223 2667 tablet_service.cc:1514] Processing DeleteTablet for tablet af1473ab1ab74b378eca3951cc648276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:24 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:46312
I20260620 23:08:24.096325 2796 tablet_replica.cc:330] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: stopping tablet replica
I20260620 23:08:24.096333 2795 tablet_replica.cc:330] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: stopping tablet replica
I20260620 23:08:24.096359 2542 tablet_service.cc:1514] Processing DeleteTablet for tablet af1473ab1ab74b378eca3951cc648276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:24 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:33110
I20260620 23:08:24.096385 2795 raft_consensus.cc:2229] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:24.096385 2796 raft_consensus.cc:2229] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:24.096424 2795 raft_consensus.cc:2258] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:24.096436 2797 tablet_replica.cc:330] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: stopping tablet replica
I20260620 23:08:24.096441 2796 raft_consensus.cc:2258] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:24.096479 2797 raft_consensus.cc:2229] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:24.096519 2797 raft_consensus.cc:2258] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:24.097008 2795 ts_tablet_manager.cc:1905] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:24.098894 2795 ts_tablet_manager.cc:1918] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:24.098975 2795 log.cc:1196] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/af1473ab1ab74b378eca3951cc648276
I20260620 23:08:24.099225 2795 ts_tablet_manager.cc:1939] T af1473ab1ab74b378eca3951cc648276 P 1dd7844373ee4ccfb9075604d269714b: Deleting consensus metadata
I20260620 23:08:24.099726 2252 catalog_manager.cc:4841] TS 1dd7844373ee4ccfb9075604d269714b (127.1.32.67:40519): tablet af1473ab1ab74b378eca3951cc648276 (table test-table [id=57b691c146154bc495eeae8997470bec]) successfully deleted
I20260620 23:08:24.099851 2797 ts_tablet_manager.cc:1905] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:24.101279 2797 ts_tablet_manager.cc:1918] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:24.101338 2797 log.cc:1196] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/af1473ab1ab74b378eca3951cc648276
I20260620 23:08:24.101537 2797 ts_tablet_manager.cc:1939] T af1473ab1ab74b378eca3951cc648276 P e432f20b7ad940ce94bb55e9ec6a92b1: Deleting consensus metadata
I20260620 23:08:24.101843 2252 catalog_manager.cc:4841] TS e432f20b7ad940ce94bb55e9ec6a92b1 (127.1.32.66:46783): tablet af1473ab1ab74b378eca3951cc648276 (table test-table [id=57b691c146154bc495eeae8997470bec]) successfully deleted
I20260620 23:08:24.102036 2796 ts_tablet_manager.cc:1905] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:24.103322 2796 ts_tablet_manager.cc:1918] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:24.103384 2796 log.cc:1196] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/af1473ab1ab74b378eca3951cc648276
I20260620 23:08:24.103593 2796 ts_tablet_manager.cc:1939] T af1473ab1ab74b378eca3951cc648276 P 7f918fa8bd8b429186675e60017c2b0f: Deleting consensus metadata
I20260620 23:08:24.103883 2252 catalog_manager.cc:4841] TS 7f918fa8bd8b429186675e60017c2b0f (127.1.32.65:35879): tablet af1473ab1ab74b378eca3951cc648276 (table test-table [id=57b691c146154bc495eeae8997470bec]) successfully deleted
Jun 20 23:08:24 dist-test-slave-fzjm krb5kdc[2220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996903, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:24.108086 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:24.097289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.1.32.65:35879)
0620 23:08:24.097330 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:24.097332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:24.097687 (+ 355us) server_negotiation.cc:407] Connection header received
0620 23:08:24.097769 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:24.097771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:24.097784 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:24.097883 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:24.098645 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.099547 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.100843 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.101161 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.101245 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:24.104847 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:24.104854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:24.104855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:24.104861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:24.106870 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:24.107275 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:24.107279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:24.107279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:24.107352 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:24.107579 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:24.107580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:24.107581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:24.107721 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:24.107858 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:24.107916 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 23:08:24.107988 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:08:24.108301 2462 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44224
I20260620 23:08:24.121366 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:24.109180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55906 (local address 127.1.32.126:45985)
0620 23:08:24.109216 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:24.109218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:24.109279 (+ 61us) server_negotiation.cc:407] Connection header received
0620 23:08:24.109348 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:24.109349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:24.109359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:24.109403 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:24.110034 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.110783 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.111774 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:24.112075 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:24.112111 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:24.118418 (+ 6307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:24.118419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:24.118420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:24.118439 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:24.120300 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:24.120612 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:24.120613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:24.120614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:24.120664 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:24.120863 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:24.120864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:24.120864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:24.121007 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:24.121107 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:24.121200 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 23:08:24.121277 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:08:24.121608 2264 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55906
I20260620 23:08:24.122258 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 2337
I20260620 23:08:24.131484 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 2477
I20260620 23:08:24.136286 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 2617
I20260620 23:08:24.141132 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 2233
2026-06-20T23:08:24Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2527 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:24 dist-test-slave-fzjm krb5kdc[2831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:24 dist-test-slave-fzjm krb5kdc[2831](info): set up 2 sockets
Jun 20 23:08:24 dist-test-slave-fzjm krb5kdc[2831](info): commencing operation
krb5kdc: starting...
W20260620 23:08:25.114204 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:25.217530 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46871
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:46871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:25.292531 2844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:25.292739 2844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:25.292773 2844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:25.293820 2844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:25.293882 2844 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 23:08:25.293913 2844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:25.293941 2844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:25.293968 2844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:25.293998 2844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:25.295390 2844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:46871
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:25.295653 2844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:25.295866 2844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:25.300499 2849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:25.300532 2850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.300544 2844 server_base.cc:1034] running on GCE node
W20260620 23:08:25.300526 2852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.300884 2844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:25.301162 2844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:25.302325 2844 hybrid_clock.cc:648] HybridClock initialized: now 1781996905302313 us; error 25 us; skew 500 ppm
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:25.304666 2844 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:25.305107 2844 webserver.cc:460] Webserver started at http://127.1.32.126:38265/ using document root <none> and password file <none>
I20260620 23:08:25.305346 2844 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:25.305411 2844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:25.305553 2844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:25.306497 2844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "6a93daf67be74b85b4027afdf4ca2213"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "5ea1f5cb09923e95fac152d697eecccc"
server_key_iv: "ffda82bce6383a15657aa3f3ffcb3b94"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.306860 2844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "6a93daf67be74b85b4027afdf4ca2213"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "5ea1f5cb09923e95fac152d697eecccc"
server_key_iv: "ffda82bce6383a15657aa3f3ffcb3b94"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.308115 2844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:25.308743 2859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.308907 2844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:25.308995 2844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "6a93daf67be74b85b4027afdf4ca2213"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "5ea1f5cb09923e95fac152d697eecccc"
server_key_iv: "ffda82bce6383a15657aa3f3ffcb3b94"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.309067 2844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:25.327639 2844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:25.329418 2844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:25.329557 2844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:25.333822 2844 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:46871
I20260620 23:08:25.333900 2921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:46871 every 8 connection(s)
I20260620 23:08:25.334189 2844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:25.334844 2922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:25.336961 2922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Bootstrap starting.
I20260620 23:08:25.337528 2922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:25.337797 2922 log.cc:824] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:25.338440 2922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: No bootstrap required, opened a new log
I20260620 23:08:25.340479 2922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a93daf67be74b85b4027afdf4ca2213" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46871 } }
I20260620 23:08:25.340619 2922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:25.340641 2922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a93daf67be74b85b4027afdf4ca2213, State: Initialized, Role: FOLLOWER
I20260620 23:08:25.340803 2922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a93daf67be74b85b4027afdf4ca2213" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46871 } }
I20260620 23:08:25.340876 2922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:25.340903 2922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:25.340942 2922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:25.341146 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 2844
I20260620 23:08:25.341246 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:25.341418 1153 external_mini_cluster.cc:1428] Setting key 748bdfe123b814bfd0eb78fcbdc4e6e6
I20260620 23:08:25.341570 2922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a93daf67be74b85b4027afdf4ca2213" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46871 } }
I20260620 23:08:25.341684 2922 leader_election.cc:302] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a93daf67be74b85b4027afdf4ca2213; no voters:
I20260620 23:08:25.341852 2922 leader_election.cc:288] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:25.341948 2925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:25.342084 2922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:25.342275 2925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [term 1 LEADER]: Becoming Leader. State: Replica: 6a93daf67be74b85b4027afdf4ca2213, State: Running, Role: LEADER
I20260620 23:08:25.342396 2925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a93daf67be74b85b4027afdf4ca2213" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46871 } }
I20260620 23:08:25.342831 2928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a93daf67be74b85b4027afdf4ca2213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a93daf67be74b85b4027afdf4ca2213" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46871 } } }
I20260620 23:08:25.342952 2928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:25.342840 2929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a93daf67be74b85b4027afdf4ca2213. Latest consensus state: current_term: 1 leader_uuid: "6a93daf67be74b85b4027afdf4ca2213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a93daf67be74b85b4027afdf4ca2213" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46871 } } }
I20260620 23:08:25.343050 2929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:25.343223 2936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:25.343904 2936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:25.345683 2936 catalog_manager.cc:1347] Generated new cluster ID: abeacef3607e4939a1edcaa846f2b4e6
I20260620 23:08:25.345733 2936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:25.352432 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.341846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.1.32.126:46871)
0620 23:08:25.342017 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.342026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.342057 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:08:25.342165 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.342173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.342226 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.343220 (+ 994us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.343349 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.344596 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.345535 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.346139 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.346235 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.348910 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.348913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.348914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.348925 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.350906 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.351259 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.351262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.351262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.351382 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.351573 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.351574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.351574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.351883 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.352044 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.352109 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.352244 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260620 23:08:25.373154 2936 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:25.373764 2936 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:25.384166 2936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Generated new TSK 0
I20260620 23:08:25.384387 2936 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:25.416004 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46871
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:25.510035 2948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:25.510215 2948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:25.510237 2948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:25.511385 2948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:25.511421 2948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:25.511439 2948 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 23:08:25.511451 2948 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:25.511490 2948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:25.512919 2948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:46871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:25.513159 2948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:25.513362 2948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:25.517971 2953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:25.517975 2956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.518221 2948 server_base.cc:1034] running on GCE node
W20260620 23:08:25.518313 2954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.518492 2948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:25.518678 2948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:25.519838 2948 hybrid_clock.cc:648] HybridClock initialized: now 1781996905519834 us; error 34 us; skew 500 ppm
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:25.522281 2948 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:25.522715 2948 webserver.cc:460] Webserver started at http://127.1.32.65:44765/ using document root <none> and password file <none>
I20260620 23:08:25.523003 2948 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:25.523074 2948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:25.523236 2948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:25.524222 2948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "15f0928cb434409aaddd4b7cbd575eb0"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "7bf7d1c3ceb049ee392ab35b498b5607"
server_key_iv: "0acacd247becee7a178a097541598649"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.524607 2948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "15f0928cb434409aaddd4b7cbd575eb0"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "7bf7d1c3ceb049ee392ab35b498b5607"
server_key_iv: "0acacd247becee7a178a097541598649"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.525928 2948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:25.526600 2963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.526784 2948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:25.526870 2948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "15f0928cb434409aaddd4b7cbd575eb0"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "7bf7d1c3ceb049ee392ab35b498b5607"
server_key_iv: "0acacd247becee7a178a097541598649"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.526978 2948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:25.537653 2948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:25.539901 2948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:25.540041 2948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:25.540495 2948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:25.540555 2948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.540596 2948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:25.540620 2948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.547328 2948 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:45537
I20260620 23:08:25.547395 3078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:45537 every 8 connection(s)
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:25.547689 2948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:25.550443 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.541002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:43149 (local address 127.1.32.126:46871)
0620 23:08:25.541074 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.541077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.541251 (+ 174us) server_negotiation.cc:407] Connection header received
0620 23:08:25.541700 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.541702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.541719 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.541801 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.542589 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.543511 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.545060 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.545399 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.545482 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.548186 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.548187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.548188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.548194 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.549329 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.549682 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.549683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.549683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.549738 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.549954 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.549955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.549955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.550060 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.550134 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.550321 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.550384 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:08:25.550576 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.540958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46871 (local address 127.1.32.65:43149)
0620 23:08:25.541113 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.541163 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.541604 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.541845 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.541872 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.542226 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:25.542485 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.542490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.543663 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.543664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.544955 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.544958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.545514 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.545515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.545661 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.546202 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:25.546213 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:25.548095 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:25.549404 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.549410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.549411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.549604 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.549790 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.549791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.549791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.549897 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.550131 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:25.550135 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:25.550234 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.550307 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.550407 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260620 23:08:25.550928 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 2948
I20260620 23:08:25.551028 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:25.551088 3079 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46871
I20260620 23:08:25.551167 3079 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:25.551199 1153 external_mini_cluster.cc:1428] Setting key 51ddfbe9e49a63c41300997163a17c2d
I20260620 23:08:25.551337 3079 heartbeater.cc:511] Master 127.1.32.126:46871 requested a full tablet report, sending...
I20260620 23:08:25.551862 2876 ts_manager.cc:194] Registered new tserver with Master: 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65:45537)
I20260620 23:08:25.552501 2876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:43149
I20260620 23:08:25.558689 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.552113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:40519 (local address 127.1.32.126:46871)
0620 23:08:25.552842 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.552844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.552853 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:25.552873 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.552874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.552884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.552955 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.553294 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.553999 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.555085 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.555263 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.555318 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.556231 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.556232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.556232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.556239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.557625 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.557988 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.557989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.557990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.558038 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.558228 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.558229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.558229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.558330 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.558398 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.558538 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.558608 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708}
I20260620 23:08:25.558704 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.552049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46871 (local address 127.1.32.65:40519)
0620 23:08:25.552172 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.552182 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.552227 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.552974 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.552975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.553147 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:25.553200 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.553202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.554117 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.554118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.554991 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.554994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.555339 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.555339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.555392 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.555823 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:25.555828 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:25.556112 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:25.557724 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.557725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.557725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.557891 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.558104 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.558105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.558105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.558159 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.558391 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:25.558391 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:25.558433 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.558516 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.558608 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:25.575587 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46871
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:25.652472 3088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:25.652678 3088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:25.652702 3088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:25.653806 3088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:25.653844 3088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:25.653865 3088 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 23:08:25.653878 3088 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:25.653901 3088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:25.655298 3088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:46871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:25.655481 3088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:25.655658 3088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:25.660375 3094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:25.660442 3093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.660490 3088 server_base.cc:1034] running on GCE node
W20260620 23:08:25.660382 3096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.660734 3088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:25.660956 3088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:25.662106 3088 hybrid_clock.cc:648] HybridClock initialized: now 1781996905662087 us; error 37 us; skew 500 ppm
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:25.664422 3088 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:25.664781 3088 webserver.cc:460] Webserver started at http://127.1.32.66:41807/ using document root <none> and password file <none>
I20260620 23:08:25.664954 3088 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:25.664991 3088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:25.665072 3088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:25.665962 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "f99abacd5b944ff0b78a2695b4cf7527"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "430656dfa32a30689e93471a1ad19b0e"
server_key_iv: "7d93cfb2e9d3b607dbb94755f309749a"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.666249 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "f99abacd5b944ff0b78a2695b4cf7527"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "430656dfa32a30689e93471a1ad19b0e"
server_key_iv: "7d93cfb2e9d3b607dbb94755f309749a"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.667619 3088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:25.668253 3103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.668417 3088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:25.668505 3088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "f99abacd5b944ff0b78a2695b4cf7527"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "430656dfa32a30689e93471a1ad19b0e"
server_key_iv: "7d93cfb2e9d3b607dbb94755f309749a"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.668574 3088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:25.689631 3088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:25.691900 3088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:25.692059 3088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:25.692530 3088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:25.692587 3088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.692626 3088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:25.692699 3088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:25.699261 3088 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:45875
I20260620 23:08:25.699378 3218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:45875 every 8 connection(s)
I20260620 23:08:25.699674 3088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:25.701812 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.693067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:38319 (local address 127.1.32.126:46871)
0620 23:08:25.693108 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.693110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.693345 (+ 235us) server_negotiation.cc:407] Connection header received
0620 23:08:25.693920 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.693922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.693935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.694003 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.694955 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.695724 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.697272 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.697564 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.697638 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.699688 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.699689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.699690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.699697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.700879 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.701186 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.701187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.701187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.701232 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.701387 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.701388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.701388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.701472 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.701513 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.701707 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.701759 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:08:25.702024 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.693055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46871 (local address 127.1.32.66:38319)
0620 23:08:25.693234 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.693264 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.693777 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.694122 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.694127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.694516 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:25.694807 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.694813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.695815 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.695816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.697153 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.697156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.697641 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.697642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.697738 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.698200 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:25.698207 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:25.699585 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:25.700939 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.700943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.700945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.701126 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.701266 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.701267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.701267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.701338 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.701520 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:25.701524 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:25.701614 (+ 90us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.701697 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.701793 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260620 23:08:25.702484 3219 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46871
I20260620 23:08:25.702553 3219 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:25.702701 3219 heartbeater.cc:511] Master 127.1.32.126:46871 requested a full tablet report, sending...
I20260620 23:08:25.703049 2875 ts_manager.cc:194] Registered new tserver with Master: f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875)
I20260620 23:08:25.703436 2875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:38319
I20260620 23:08:25.707947 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.703395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46871 (local address 127.1.32.66:56205)
0620 23:08:25.703617 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.703627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.703690 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.703830 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.703831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.703967 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:25.704010 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.704011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.704678 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.704678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.705129 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.705130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.705410 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.705411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.705458 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.705779 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:25.705782 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:25.706037 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:25.707173 (+ 1136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.707173 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.707174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.707309 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.707468 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.707469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.707469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.707512 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.707748 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:25.707748 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:25.707780 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.707828 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.707894 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":74,"threads_started":1}
I20260620 23:08:25.707948 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.703474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:56205 (local address 127.1.32.126:46871)
0620 23:08:25.703509 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.703510 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.703699 (+ 189us) server_negotiation.cc:407] Connection header received
0620 23:08:25.703754 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.703754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.703763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.703828 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.704075 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.704579 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.705192 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.705351 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.705396 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.706100 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.706101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.706101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.706106 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.707114 (+ 1008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.707366 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.707366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.707366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.707407 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.707551 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.707551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.707551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.707698 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.707743 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.707841 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.707897 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:08:25.709851 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 3088
I20260620 23:08:25.709919 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:25.710038 1153 external_mini_cluster.cc:1428] Setting key 692c7cf589001a42b4b96d3030fbb124
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:25.731500 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46871
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:25.807520 3228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:25.807693 3228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:25.807714 3228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:25.808655 3228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:25.808699 3228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:25.808717 3228 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 23:08:25.808727 3228 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:25.808743 3228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:25.810000 3228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:46871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:25.810202 3228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:25.810385 3228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:25.814994 3234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:25.814983 3236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.815068 3228 server_base.cc:1034] running on GCE node
W20260620 23:08:25.814982 3233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:25.815383 3228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:25.815634 3228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:25.816780 3228 hybrid_clock.cc:648] HybridClock initialized: now 1781996905816764 us; error 30 us; skew 500 ppm
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:25.818981 3228 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:25.819324 3228 webserver.cc:460] Webserver started at http://127.1.32.67:41483/ using document root <none> and password file <none>
I20260620 23:08:25.819545 3228 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:25.819583 3228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:25.819718 3228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:25.820574 3228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "8c7555c658e4924df93d82a466b713c0"
server_key_iv: "999ad9284f854edab3f3ab8ebb6a99dd"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.820911 3228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "8c7555c658e4924df93d82a466b713c0"
server_key_iv: "999ad9284f854edab3f3ab8ebb6a99dd"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.822100 3228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:25.822777 3243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.822971 3228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:25.823037 3228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
format_stamp: "Formatted at 2026-06-20 23:08:25 on dist-test-slave-fzjm"
server_key: "8c7555c658e4924df93d82a466b713c0"
server_key_iv: "999ad9284f854edab3f3ab8ebb6a99dd"
server_key_version: "encryptionkey@0"
I20260620 23:08:25.823119 3228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:25.831387 3228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:25.833001 3228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:25.833132 3228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:25.833526 3228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:25.833580 3228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.833617 3228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:25.833638 3228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.839949 3228 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39213
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:25.840011 3358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39213 every 8 connection(s)
I20260620 23:08:25.840262 3228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:08:25.843269 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.834039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:45649 (local address 127.1.32.126:46871)
0620 23:08:25.834076 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.834078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.834273 (+ 195us) server_negotiation.cc:407] Connection header received
0620 23:08:25.834737 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.834738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.834751 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.834805 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.835552 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.836305 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.837730 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.838049 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.838132 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.840411 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.840413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.840413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.840420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.841900 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.842281 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.842282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.842282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.842337 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.842581 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.842582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.842582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.842717 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.842798 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.843111 (+ 313us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.843181 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:25.843421 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.834033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46871 (local address 127.1.32.67:45649)
0620 23:08:25.834172 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.834200 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.834633 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.834876 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.834879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.835215 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:25.835456 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.835461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.836407 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.836408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.837626 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.837629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.838142 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.838143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.838308 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.838767 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:25.838777 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:25.840321 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:25.841978 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.841984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.841986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.842201 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.842410 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.842410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.842411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.842515 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.842790 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:25.842795 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:25.842960 (+ 165us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.843088 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.843230 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260620 23:08:25.843935 3359 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46871
I20260620 23:08:25.844012 3359 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:25.844189 3359 heartbeater.cc:511] Master 127.1.32.126:46871 requested a full tablet report, sending...
I20260620 23:08:25.844511 2876 ts_manager.cc:194] Registered new tserver with Master: f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:25.844993 2876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:45649
I20260620 23:08:25.845513 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 3228
I20260620 23:08:25.845597 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:25.845739 1153 external_mini_cluster.cc:1428] Setting key a65f7fec72ceb867d317a88e4c9d39ea
I20260620 23:08:25.846508 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:08:25.850431 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.844870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:58907 (local address 127.1.32.126:46871)
0620 23:08:25.844908 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.844909 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.844975 (+ 66us) server_negotiation.cc:407] Connection header received
0620 23:08:25.845043 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.845044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.845051 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.845115 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.845469 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.846092 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.846863 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.847094 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.847167 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.847978 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.847979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.847979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.847985 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.849398 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.849722 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.849723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.849723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.849771 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.849963 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.849964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.849964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.850079 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.850145 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.850286 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.850351 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:25.850436 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.844790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46871 (local address 127.1.32.67:58907)
0620 23:08:25.844915 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.844925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.844981 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.845119 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.845120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.845304 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:25.845392 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.845394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.846164 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.846164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.846760 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.846762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.847166 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.847167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.847242 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.847629 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:25.847632 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:25.847884 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:25.849481 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.849482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.849482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.849641 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.849840 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.849841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.849841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.849894 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.850138 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:25.850139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:25.850185 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.850264 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.850352 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:25.868286 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.861962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.1.32.126:46871)
0620 23:08:25.862004 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.862007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.862015 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:25.862076 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.862077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.862087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.862139 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.862628 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.863182 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.863880 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.864059 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.864129 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.866108 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.866110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.866110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.866117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.867234 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.867563 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.867564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.867564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.867624 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.867815 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.867816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.867816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.867909 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.867999 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.868092 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.868159 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:08:25.869628 2876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:25.871340 2876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:25.877377 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.873330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45875 (local address 127.0.0.1:39376)
0620 23:08:25.873732 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.873740 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.873796 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.873953 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.873953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.873955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:25.874017 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.874018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.875326 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.875327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.876289 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.876291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.877084 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.877085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.877169 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.877183 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.877258 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.877304 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":16,"thread_start_us":46,"threads_started":1}
I20260620 23:08:25.877409 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.873483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.1.32.66:45875)
0620 23:08:25.873646 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.873650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.873803 (+ 153us) server_negotiation.cc:407] Connection header received
0620 23:08:25.873860 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.873862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.873877 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.873947 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:25.874176 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.875236 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.876372 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.877001 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.877070 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.877120 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.877281 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.877318 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":62,"threads_started":1}
I20260620 23:08:25.877722 3373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.873330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39213 (local address 127.0.0.1:38390)
0620 23:08:25.873468 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.873500 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.873558 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.873935 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.873938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.873940 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:25.874017 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.874020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.875341 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.875342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.876687 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.876688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.877485 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.877487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.877566 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.877570 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.877608 (+ 38us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.877655 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":113,"thread_start_us":45,"threads_started":1}
I20260620 23:08:25.877849 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.873357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:45537 (local address 127.0.0.1:50168)
0620 23:08:25.873980 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.873991 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.874042 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.874331 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.874332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.874334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:25.874377 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.874378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.875312 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.875313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.876684 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.876687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.877624 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.877625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.877664 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.877668 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.877747 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.877784 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":58,"thread_start_us":57,"threads_started":1}
I20260620 23:08:25.877889 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.873559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.1.32.65:45537)
0620 23:08:25.873694 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.873698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.874038 (+ 340us) server_negotiation.cc:407] Connection header received
0620 23:08:25.874174 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.874177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.874194 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.874265 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:25.874467 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.875232 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.876768 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.877466 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.877531 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.877585 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.877762 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.877795 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260620 23:08:25.878165 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.873546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.1.32.67:39213)
0620 23:08:25.873726 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.873731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.873744 (+ 13us) server_negotiation.cc:407] Connection header received
0620 23:08:25.873793 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.873796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.873813 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.873905 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:25.874103 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.875241 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.876780 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.877392 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.877855 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.877973 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.877997 (+ 24us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.878059 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260620 23:08:25.878410 3013 tablet_service.cc:1467] Processing CreateTablet for tablet c45f72e45ada41ed8978be9d6601648b (DEFAULT_TABLE table=test-table [id=3e009e3eead44bdd80c81fa99cfd141b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:25.878398 3153 tablet_service.cc:1467] Processing CreateTablet for tablet c45f72e45ada41ed8978be9d6601648b (DEFAULT_TABLE table=test-table [id=3e009e3eead44bdd80c81fa99cfd141b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:25.878749 3153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45f72e45ada41ed8978be9d6601648b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:25.878748 3013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45f72e45ada41ed8978be9d6601648b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:25.878726 3293 tablet_service.cc:1467] Processing CreateTablet for tablet c45f72e45ada41ed8978be9d6601648b (DEFAULT_TABLE table=test-table [id=3e009e3eead44bdd80c81fa99cfd141b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:25.879050 3293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45f72e45ada41ed8978be9d6601648b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:25.881291 3379 tablet_bootstrap.cc:492] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Bootstrap starting.
I20260620 23:08:25.882083 3379 tablet_bootstrap.cc:654] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:25.882162 3381 tablet_bootstrap.cc:492] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Bootstrap starting.
I20260620 23:08:25.882601 3379 log.cc:824] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:25.882720 3381 tablet_bootstrap.cc:654] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:25.883033 3380 tablet_bootstrap.cc:492] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Bootstrap starting.
I20260620 23:08:25.883081 3381 log.cc:824] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:25.883349 3379 tablet_bootstrap.cc:492] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: No bootstrap required, opened a new log
I20260620 23:08:25.883435 3379 ts_tablet_manager.cc:1397] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:25.883566 3380 tablet_bootstrap.cc:654] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:25.883901 3381 tablet_bootstrap.cc:492] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: No bootstrap required, opened a new log
I20260620 23:08:25.883904 3380 log.cc:824] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:25.883967 3381 ts_tablet_manager.cc:1397] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:25.884724 3380 tablet_bootstrap.cc:492] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: No bootstrap required, opened a new log
I20260620 23:08:25.884793 3380 ts_tablet_manager.cc:1397] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:25.885388 3379 raft_consensus.cc:348] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.885529 3379 raft_consensus.cc:374] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:25.885578 3379 raft_consensus.cc:729] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51a3de1cbbc48f6a63c1fedaf4ced6e, State: Initialized, Role: FOLLOWER
I20260620 23:08:25.885689 3379 consensus_queue.cc:260] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.885944 3379 ts_tablet_manager.cc:1428] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:08:25.885958 3359 heartbeater.cc:503] Master 127.1.32.126:46871 was elected leader, sending a full tablet report...
I20260620 23:08:25.885905 3381 raft_consensus.cc:348] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.886053 3381 raft_consensus.cc:374] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:25.886088 3381 raft_consensus.cc:729] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99abacd5b944ff0b78a2695b4cf7527, State: Initialized, Role: FOLLOWER
I20260620 23:08:25.886183 3381 consensus_queue.cc:260] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.886420 3381 ts_tablet_manager.cc:1428] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:08:25.886554 3219 heartbeater.cc:503] Master 127.1.32.126:46871 was elected leader, sending a full tablet report...
I20260620 23:08:25.886950 3380 raft_consensus.cc:348] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.887095 3380 raft_consensus.cc:374] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:25.887133 3380 raft_consensus.cc:729] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Initialized, Role: FOLLOWER
I20260620 23:08:25.887246 3380 consensus_queue.cc:260] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.887496 3380 ts_tablet_manager.cc:1428] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:25.887527 3079 heartbeater.cc:503] Master 127.1.32.126:46871 was elected leader, sending a full tablet report...
I20260620 23:08:25.925536 3387 raft_consensus.cc:482] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:25.925627 3387 raft_consensus.cc:504] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.926004 3387 leader_election.cc:288] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:25.928951 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.926258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:47783 (local address 127.1.32.66:45875)
0620 23:08:25.926301 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.926303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.926499 (+ 196us) server_negotiation.cc:407] Connection header received
0620 23:08:25.926528 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.926529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.926540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.926597 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:25.926761 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.927253 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.928213 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.928583 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.928638 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.928647 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.928848 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.928884 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:08:25.928963 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.926189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45875 (local address 127.1.32.65:47783)
0620 23:08:25.926420 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.926430 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.926485 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.926613 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.926613 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.926615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:25.926688 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.926690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.927335 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.927336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.928129 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.928131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.928671 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.928672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.928746 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.928750 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.928837 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.928874 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":69,"thread_start_us":102,"threads_started":1}
I20260620 23:08:25.929144 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.926189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39213 (local address 127.1.32.65:38935)
0620 23:08:25.926279 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.926291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.926342 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.926507 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.926508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.926528 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:25.926605 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.926607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.927330 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.927331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.928211 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.928213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.928875 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.928876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.928960 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.928963 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.929021 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.929062 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":59}
I20260620 23:08:25.929234 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.926333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:38935 (local address 127.1.32.67:39213)
0620 23:08:25.926368 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.926370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.926379 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:25.926404 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.926405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.926416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.926485 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:25.926718 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.927251 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.928381 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.928797 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.928857 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.928866 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.929104 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.929154 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:25.929495 3313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f72e45ada41ed8978be9d6601648b" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" is_pre_election: true
I20260620 23:08:25.929513 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f72e45ada41ed8978be9d6601648b" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527" is_pre_election: true
I20260620 23:08:25.929664 3313 raft_consensus.cc:2454] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:25.929674 3173 raft_consensus.cc:2454] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:25.929876 2964 leader_election.cc:302] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f99abacd5b944ff0b78a2695b4cf7527; no voters:
I20260620 23:08:25.930004 3387 raft_consensus.cc:2781] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:25.930051 3387 raft_consensus.cc:482] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:25.930080 3387 raft_consensus.cc:3037] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:25.930720 3387 raft_consensus.cc:504] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.930851 3387 leader_election.cc:288] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Requested vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:25.930994 3313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f72e45ada41ed8978be9d6601648b" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
I20260620 23:08:25.931005 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45f72e45ada41ed8978be9d6601648b" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527"
I20260620 23:08:25.931063 3313 raft_consensus.cc:3037] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:25.931087 3173 raft_consensus.cc:3037] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:25.931675 3313 raft_consensus.cc:2454] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:25.931726 3173 raft_consensus.cc:2454] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:25.931849 2964 leader_election.cc:302] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f99abacd5b944ff0b78a2695b4cf7527; no voters:
I20260620 23:08:25.931942 3387 raft_consensus.cc:2781] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:25.932056 3387 raft_consensus.cc:686] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 LEADER]: Becoming Leader. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Running, Role: LEADER
I20260620 23:08:25.932138 3387 consensus_queue.cc:237] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } }
I20260620 23:08:25.932874 2875 catalog_manager.cc:5495] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65). New cstate: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } health_report { overall_health: HEALTHY } } }
W20260620 23:08:25.950455 3220 tablet.cc:2369] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 23:08:25 dist-test-slave-fzjm krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:25.957854 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.951449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46871 (local address 127.0.0.1:32794)
0620 23:08:25.951622 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:25.951630 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:25.951687 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:25.951875 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:25.951876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:25.952193 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:25.952275 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.952277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.952877 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.952877 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.953389 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:25.953390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.953717 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.953718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:25.953757 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.954292 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:25.954296 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:25.955592 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:25.956978 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.956983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.956985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.957160 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.957327 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:25.957327 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:25.957328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:25.957378 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:25.957572 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:25.957579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:25.957617 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 23:08:25.957667 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:08:25.957768 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260620 23:08:25.957922 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:25.951524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.1.32.126:46871)
0620 23:08:25.951576 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:25.951578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:25.951710 (+ 132us) server_negotiation.cc:407] Connection header received
0620 23:08:25.951735 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:25.951736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:25.951744 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:25.951797 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:25.952345 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.952808 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.953456 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:25.953625 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:25.953672 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:25.955661 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:25.955662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:25.955662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:25.955667 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:25.956901 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.957224 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.957225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.957225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.957270 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:25.957429 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:25.957429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:25.957429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:25.957519 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:25.957567 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:25.957702 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:08:25.957811 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":30}
I20260620 23:08:25.959847 2875 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:32794:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 23:08:25.960071 2875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:25.961571 3293 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab1e5a058054140986ec9d5ac55263d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0b93649ac744f37a8880ab1de0500e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:25.961759 3293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab1e5a058054140986ec9d5ac55263d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:25.961726 3013 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab1e5a058054140986ec9d5ac55263d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0b93649ac744f37a8880ab1de0500e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:25.961899 3013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab1e5a058054140986ec9d5ac55263d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:25.962929 3379 tablet_bootstrap.cc:492] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: Bootstrap starting.
I20260620 23:08:25.962965 3380 tablet_bootstrap.cc:492] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: Bootstrap starting.
I20260620 23:08:25.963092 3153 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab1e5a058054140986ec9d5ac55263d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0b93649ac744f37a8880ab1de0500e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:25.963253 3153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab1e5a058054140986ec9d5ac55263d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:25.963333 3380 tablet_bootstrap.cc:654] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:25.963486 3379 tablet_bootstrap.cc:654] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:25.963912 3380 tablet_bootstrap.cc:492] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: No bootstrap required, opened a new log
I20260620 23:08:25.963949 3380 ts_tablet_manager.cc:1397] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:25.964110 3380 raft_consensus.cc:348] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:25.964172 3380 raft_consensus.cc:374] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:25.964186 3380 raft_consensus.cc:729] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Initialized, Role: FOLLOWER
I20260620 23:08:25.964206 3380 consensus_queue.cc:260] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:25.964313 3380 ts_tablet_manager.cc:1428] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.964350 3387 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:25.964448 3387 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:25.964735 3381 tablet_bootstrap.cc:492] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: Bootstrap starting.
I20260620 23:08:25.965369 3379 tablet_bootstrap.cc:492] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: No bootstrap required, opened a new log
I20260620 23:08:25.965421 3379 ts_tablet_manager.cc:1397] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 23:08:25.965502 3381 tablet_bootstrap.cc:654] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:25.965565 3379 raft_consensus.cc:348] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:25.965638 3379 raft_consensus.cc:374] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:25.965664 3379 raft_consensus.cc:729] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51a3de1cbbc48f6a63c1fedaf4ced6e, State: Initialized, Role: FOLLOWER
I20260620 23:08:25.965706 3379 consensus_queue.cc:260] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:25.965791 3379 ts_tablet_manager.cc:1428] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.965862 3385 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:25.966019 3385 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:25.966120 3381 tablet_bootstrap.cc:492] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: No bootstrap required, opened a new log
I20260620 23:08:25.966172 3381 ts_tablet_manager.cc:1397] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:25.966318 3381 raft_consensus.cc:348] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:25.966385 3381 raft_consensus.cc:374] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:25.966413 3381 raft_consensus.cc:729] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99abacd5b944ff0b78a2695b4cf7527, State: Initialized, Role: FOLLOWER
I20260620 23:08:25.966452 3381 consensus_queue.cc:260] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:25.966529 3381 ts_tablet_manager.cc:1428] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:25.966599 3386 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:25.966725 3386 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 23:08:26.048409 3080 tablet.cc:2369] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:26.075160 3387 raft_consensus.cc:482] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:26.075261 3387 raft_consensus.cc:504] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.075459 3387 leader_election.cc:288] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:26.075644 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab1e5a058054140986ec9d5ac55263d" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527" is_pre_election: true
I20260620 23:08:26.075732 3173 raft_consensus.cc:2454] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:26.075738 3313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab1e5a058054140986ec9d5ac55263d" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" is_pre_election: true
I20260620 23:08:26.075852 3313 raft_consensus.cc:2454] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:26.075881 2964 leader_election.cc:302] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f99abacd5b944ff0b78a2695b4cf7527; no voters:
I20260620 23:08:26.075969 3387 raft_consensus.cc:2781] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:26.076009 3387 raft_consensus.cc:482] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:26.076040 3387 raft_consensus.cc:3037] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:26.076697 3387 raft_consensus.cc:504] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.076833 3387 leader_election.cc:288] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Requested vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:26.076999 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab1e5a058054140986ec9d5ac55263d" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527"
I20260620 23:08:26.077001 3313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab1e5a058054140986ec9d5ac55263d" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
I20260620 23:08:26.077064 3173 raft_consensus.cc:3037] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:26.077087 3313 raft_consensus.cc:3037] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:26.077760 3313 raft_consensus.cc:2454] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:26.077852 3173 raft_consensus.cc:2454] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:26.077898 2966 leader_election.cc:302] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f51a3de1cbbc48f6a63c1fedaf4ced6e; no voters:
I20260620 23:08:26.077975 3387 raft_consensus.cc:2781] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:26.078018 3387 raft_consensus.cc:686] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 LEADER]: Becoming Leader. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Running, Role: LEADER
I20260620 23:08:26.078065 3387 consensus_queue.cc:237] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.078354 3389 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: TxnStatusTablet state changed. Reason: New leader 15f0928cb434409aaddd4b7cbd575eb0. Latest consensus state: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:26.078433 3389 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:26.078574 3401 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 23:08:26.078722 3401 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 23:08:26.078622 2876 catalog_manager.cc:5495] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65). New cstate: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } health_report { overall_health: UNKNOWN } } }
W20260620 23:08:26.090826 3360 tablet.cc:2369] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:26.111781 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:26.109027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:45537 (local address 127.0.0.1:50178)
0620 23:08:26.109108 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:26.109120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:26.109178 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:26.109342 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:26.109342 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:26.109343 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:26.109386 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:26.109387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.110226 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.110227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:26.110734 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:26.110735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.111211 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.111212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:26.111304 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:26.111320 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:26.111624 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:26.111632 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 23:08:26.111677 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 23:08:26.111725 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 23:08:26.111789 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:26.109119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.1.32.65:45537)
0620 23:08:26.109187 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:26.109189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:26.109220 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:08:26.109253 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:26.109254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:26.109274 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:26.109334 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:26.109451 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.110144 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.110815 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.111099 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.111185 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:26.111416 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:26.111532 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:26.111609 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:26.111689 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:08:26.111711 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:08:26.113682 3313 raft_consensus.cc:1261] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:26.113739 3173 raft_consensus.cc:1261] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:26.114019 3387 consensus_queue.cc:1035] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:26.114255 3387 consensus_queue.cc:1035] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:26.115343 3386 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: TxnStatusTablet state changed. Reason: New leader 15f0928cb434409aaddd4b7cbd575eb0. Latest consensus state: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:26.115450 3386 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:26.115687 3385 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: TxnStatusTablet state changed. Reason: New leader 15f0928cb434409aaddd4b7cbd575eb0. Latest consensus state: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:26.115787 3385 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:26.116475 3387 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:26.116554 3387 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:26.116997 3385 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:26.117089 3385 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P f51a3de1cbbc48f6a63c1fedaf4ced6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:26.117117 3404 mvcc.cc:205] Tried to move back new op lower bound from 7299059327439286272 to 7299059327296131072. Current Snapshot: MvccSnapshot[applied={T|T < 7299059327439286272}]
I20260620 23:08:26.117209 3407 mvcc.cc:205] Tried to move back new op lower bound from 7299059327439286272 to 7299059327296131072. Current Snapshot: MvccSnapshot[applied={T|T < 7299059327439286272}]
I20260620 23:08:26.118299 3389 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:26.118391 3389 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P 15f0928cb434409aaddd4b7cbd575eb0: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:26.118712 3386 tablet_replica.cc:439] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } } }
I20260620 23:08:26.118804 3386 tablet_replica.cc:442] T 3ab1e5a058054140986ec9d5ac55263d P f99abacd5b944ff0b78a2695b4cf7527: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:26.122143 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:26.119700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.1.32.65:45537)
0620 23:08:26.119882 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:26.119883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:26.119890 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:26.119908 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:26.119909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:26.119917 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:26.119968 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:26.120105 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.120818 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.121493 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.121784 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.121850 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:26.121950 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:26.121979 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:26.122022 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:26.122063 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 23:08:26.122080 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260620 23:08:26.127298 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:26.124536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:45537 (local address 127.1.32.65:57017)
0620 23:08:26.124715 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:26.124725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:26.124788 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:26.124913 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:26.124914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:26.124915 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:26.124959 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:26.124960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.125767 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.125768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:26.126442 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:26.126444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.126880 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.126881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:26.126999 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:26.127011 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:26.127172 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:26.127173 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:08:26.127199 (+ 26us) client_negotiation.cc:240] Negotiation successful
0620 23:08:26.127227 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260620 23:08:26.127306 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:26.124598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57017 (local address 127.1.32.65:45537)
0620 23:08:26.124656 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:26.124658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:26.124798 (+ 140us) server_negotiation.cc:407] Connection header received
0620 23:08:26.124824 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:26.124824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:26.124833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:26.124912 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:26.125036 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.125696 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.126517 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:26.126806 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:26.126872 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:26.127092 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:26.127121 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:26.127172 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:26.127213 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 23:08:26.127230 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:08:26.129171 3313 raft_consensus.cc:1261] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:26.129201 3173 raft_consensus.cc:1261] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:26.129400 3411 consensus_queue.cc:1035] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:26.129503 3389 consensus_queue.cc:1035] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:26.139952 2875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:26.140055 2875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:26.140776 2875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Sending DeleteTablet for 3 replicas of tablet c45f72e45ada41ed8978be9d6601648b
I20260620 23:08:26.141070 3293 tablet_service.cc:1514] Processing DeleteTablet for tablet c45f72e45ada41ed8978be9d6601648b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:26 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260620 23:08:26.141059 3153 tablet_service.cc:1514] Processing DeleteTablet for tablet c45f72e45ada41ed8978be9d6601648b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:26 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39376
I20260620 23:08:26.141183 3013 tablet_service.cc:1514] Processing DeleteTablet for tablet c45f72e45ada41ed8978be9d6601648b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:26 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:50168
I20260620 23:08:26.141273 3420 tablet_replica.cc:330] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: stopping tablet replica
I20260620 23:08:26.141403 3422 tablet_replica.cc:330] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: stopping tablet replica
I20260620 23:08:26.141409 3420 raft_consensus.cc:2229] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:26.141469 3420 raft_consensus.cc:2258] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:26.141462 2875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 23:08:26.141566 3422 raft_consensus.cc:2229] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 LEADER]: Raft consensus shutting down.
W20260620 23:08:26.141593 2875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:26.141646 3422 raft_consensus.cc:2258] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:26.141638 3421 tablet_replica.cc:330] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: stopping tablet replica
I20260620 23:08:26.141727 3421 raft_consensus.cc:2229] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:26.141796 3421 raft_consensus.cc:2258] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:26.142097 3422 ts_tablet_manager.cc:1905] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:26.142235 3421 ts_tablet_manager.cc:1905] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:26.143400 3153 tablet_service.cc:1467] Processing CreateTablet for tablet 3753ecf67eef4e039ea991df81b3e659 (DEFAULT_TABLE table=test-table [id=c2433324dcf3453ea17f947184574740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:26.143501 3153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3753ecf67eef4e039ea991df81b3e659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:26.143812 3422 ts_tablet_manager.cc:1918] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:26.143886 3422 log.cc:1196] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/c45f72e45ada41ed8978be9d6601648b
I20260620 23:08:26.144161 3422 ts_tablet_manager.cc:1939] T c45f72e45ada41ed8978be9d6601648b P 15f0928cb434409aaddd4b7cbd575eb0: Deleting consensus metadata
I20260620 23:08:26.144181 3421 ts_tablet_manager.cc:1918] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:26.144243 3421 log.cc:1196] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/c45f72e45ada41ed8978be9d6601648b
I20260620 23:08:26.144490 3421 ts_tablet_manager.cc:1939] T c45f72e45ada41ed8978be9d6601648b P f99abacd5b944ff0b78a2695b4cf7527: Deleting consensus metadata
I20260620 23:08:26.144862 3381 tablet_bootstrap.cc:492] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: Bootstrap starting.
I20260620 23:08:26.144937 2860 catalog_manager.cc:4841] TS f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875): tablet c45f72e45ada41ed8978be9d6601648b (table test-table [id=3e009e3eead44bdd80c81fa99cfd141b]) successfully deleted
I20260620 23:08:26.145048 2861 catalog_manager.cc:4841] TS 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65:45537): tablet c45f72e45ada41ed8978be9d6601648b (table test-table [id=3e009e3eead44bdd80c81fa99cfd141b]) successfully deleted
I20260620 23:08:26.145248 3381 tablet_bootstrap.cc:654] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:26.145289 3293 tablet_service.cc:1467] Processing CreateTablet for tablet 3753ecf67eef4e039ea991df81b3e659 (DEFAULT_TABLE table=test-table [id=c2433324dcf3453ea17f947184574740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:26.145387 3293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3753ecf67eef4e039ea991df81b3e659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:26.145876 3381 tablet_bootstrap.cc:492] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: No bootstrap required, opened a new log
I20260620 23:08:26.145915 3381 ts_tablet_manager.cc:1397] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:26.146035 3381 raft_consensus.cc:348] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.146092 3381 raft_consensus.cc:374] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:26.146113 3381 raft_consensus.cc:729] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99abacd5b944ff0b78a2695b4cf7527, State: Initialized, Role: FOLLOWER
I20260620 23:08:26.146163 3381 consensus_queue.cc:260] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.146245 3381 ts_tablet_manager.cc:1428] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:26.146471 3379 tablet_bootstrap.cc:492] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Bootstrap starting.
I20260620 23:08:26.146826 3379 tablet_bootstrap.cc:654] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:26.147161 3013 tablet_service.cc:1467] Processing CreateTablet for tablet 3753ecf67eef4e039ea991df81b3e659 (DEFAULT_TABLE table=test-table [id=c2433324dcf3453ea17f947184574740]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:26.147257 3013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3753ecf67eef4e039ea991df81b3e659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:26.147580 3420 ts_tablet_manager.cc:1905] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:26.148139 3380 tablet_bootstrap.cc:492] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: Bootstrap starting.
I20260620 23:08:26.148417 3380 tablet_bootstrap.cc:654] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:26.149040 3380 tablet_bootstrap.cc:492] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: No bootstrap required, opened a new log
I20260620 23:08:26.149063 3420 ts_tablet_manager.cc:1918] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:26.149103 3380 ts_tablet_manager.cc:1397] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:26.149121 3420 log.cc:1196] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/c45f72e45ada41ed8978be9d6601648b
I20260620 23:08:26.149231 3380 raft_consensus.cc:348] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.149274 3380 raft_consensus.cc:374] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:26.149286 3380 raft_consensus.cc:729] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Initialized, Role: FOLLOWER
I20260620 23:08:26.149335 3380 consensus_queue.cc:260] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.149377 3420 ts_tablet_manager.cc:1939] T c45f72e45ada41ed8978be9d6601648b P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting consensus metadata
I20260620 23:08:26.149415 3380 ts_tablet_manager.cc:1428] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:26.149761 2862 catalog_manager.cc:4841] TS f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213): tablet c45f72e45ada41ed8978be9d6601648b (table test-table [id=3e009e3eead44bdd80c81fa99cfd141b]) successfully deleted
I20260620 23:08:26.151219 3379 tablet_bootstrap.cc:492] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: No bootstrap required, opened a new log
I20260620 23:08:26.151263 3379 ts_tablet_manager.cc:1397] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260620 23:08:26.151401 3379 raft_consensus.cc:348] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.151459 3379 raft_consensus.cc:374] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:26.151473 3379 raft_consensus.cc:729] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51a3de1cbbc48f6a63c1fedaf4ced6e, State: Initialized, Role: FOLLOWER
I20260620 23:08:26.151530 3379 consensus_queue.cc:260] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.151612 3379 ts_tablet_manager.cc:1428] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:26.208577 3389 raft_consensus.cc:482] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:26.208671 3389 raft_consensus.cc:504] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.208849 3389 leader_election.cc:288] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:26.209051 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3753ecf67eef4e039ea991df81b3e659" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527" is_pre_election: true
I20260620 23:08:26.209084 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3753ecf67eef4e039ea991df81b3e659" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" is_pre_election: true
I20260620 23:08:26.209154 3173 raft_consensus.cc:2454] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:26.209184 3312 raft_consensus.cc:2454] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:26.209352 2966 leader_election.cc:302] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f51a3de1cbbc48f6a63c1fedaf4ced6e; no voters:
I20260620 23:08:26.209461 3389 raft_consensus.cc:2781] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:26.209501 3389 raft_consensus.cc:482] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:26.209527 3389 raft_consensus.cc:3037] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:26.210137 3389 raft_consensus.cc:504] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.210281 3389 leader_election.cc:288] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Requested vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:26.210417 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3753ecf67eef4e039ea991df81b3e659" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
I20260620 23:08:26.210498 3312 raft_consensus.cc:3037] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:26.210502 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3753ecf67eef4e039ea991df81b3e659" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527"
I20260620 23:08:26.210567 3173 raft_consensus.cc:3037] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:26.211333 3173 raft_consensus.cc:2454] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:26.211500 2964 leader_election.cc:302] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f99abacd5b944ff0b78a2695b4cf7527; no voters:
I20260620 23:08:26.211587 3389 raft_consensus.cc:2781] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:26.211647 3389 raft_consensus.cc:686] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 LEADER]: Becoming Leader. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Running, Role: LEADER
I20260620 23:08:26.211702 3312 raft_consensus.cc:2454] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:26.211690 3389 consensus_queue.cc:237] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:26.212258 2875 catalog_manager.cc:5495] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65). New cstate: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:26.225069 3173 raft_consensus.cc:1261] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:26.225171 3312 raft_consensus.cc:1261] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:26.225260 3411 consensus_queue.cc:1035] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:26.225361 3389 consensus_queue.cc:1035] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:08:26.229986 3404 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 23:08:27.746747 2875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:27.746850 2875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:27.747645 2875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Sending DeleteTablet for 3 replicas of tablet 3753ecf67eef4e039ea991df81b3e659
I20260620 23:08:27.747854 3013 tablet_service.cc:1514] Processing DeleteTablet for tablet 3753ecf67eef4e039ea991df81b3e659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:27 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:50168
I20260620 23:08:27.747901 3153 tablet_service.cc:1514] Processing DeleteTablet for tablet 3753ecf67eef4e039ea991df81b3e659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:27 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39376
I20260620 23:08:27.747901 3293 tablet_service.cc:1514] Processing DeleteTablet for tablet 3753ecf67eef4e039ea991df81b3e659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:27 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260620 23:08:27.748127 3449 tablet_replica.cc:330] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: stopping tablet replica
I20260620 23:08:27.748198 3449 raft_consensus.cc:2229] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:27.748232 3449 raft_consensus.cc:2258] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:27.748329 3450 tablet_replica.cc:330] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: stopping tablet replica
I20260620 23:08:27.748389 3450 raft_consensus.cc:2229] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:27.748423 3450 raft_consensus.cc:2258] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:27.748631 3450 ts_tablet_manager.cc:1905] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:27.748673 3449 ts_tablet_manager.cc:1905] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:27.750124 3450 ts_tablet_manager.cc:1918] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:08:27.750239 3450 log.cc:1196] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/3753ecf67eef4e039ea991df81b3e659
I20260620 23:08:27.750516 3450 ts_tablet_manager.cc:1939] T 3753ecf67eef4e039ea991df81b3e659 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting consensus metadata
I20260620 23:08:27.750517 3449 ts_tablet_manager.cc:1918] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:08:27.750576 3449 log.cc:1196] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/3753ecf67eef4e039ea991df81b3e659
I20260620 23:08:27.750804 3449 ts_tablet_manager.cc:1939] T 3753ecf67eef4e039ea991df81b3e659 P f99abacd5b944ff0b78a2695b4cf7527: Deleting consensus metadata
I20260620 23:08:27.750810 2862 catalog_manager.cc:4841] TS f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213): tablet 3753ecf67eef4e039ea991df81b3e659 (table test-table [id=c2433324dcf3453ea17f947184574740]) successfully deleted
I20260620 23:08:27.750973 3448 tablet_replica.cc:330] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: stopping tablet replica
I20260620 23:08:27.751051 3448 raft_consensus.cc:2229] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:27.751113 3448 raft_consensus.cc:2258] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:27.751104 2860 catalog_manager.cc:4841] TS f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875): tablet 3753ecf67eef4e039ea991df81b3e659 (table test-table [id=c2433324dcf3453ea17f947184574740]) successfully deleted
I20260620 23:08:27.751358 3448 ts_tablet_manager.cc:1905] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:27.752599 3448 ts_tablet_manager.cc:1918] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:08:27.752671 3448 log.cc:1196] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/3753ecf67eef4e039ea991df81b3e659
I20260620 23:08:27.752890 3448 ts_tablet_manager.cc:1939] T 3753ecf67eef4e039ea991df81b3e659 P 15f0928cb434409aaddd4b7cbd575eb0: Deleting consensus metadata
I20260620 23:08:27.753171 2861 catalog_manager.cc:4841] TS 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65:45537): tablet 3753ecf67eef4e039ea991df81b3e659 (table test-table [id=c2433324dcf3453ea17f947184574740]) successfully deleted
Jun 20 23:08:27 dist-test-slave-fzjm krb5kdc[2831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996907, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:08:27 dist-test-slave-fzjm krb5kdc[2831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996907, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:27.770205 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:27.763120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.1.32.126:46871)
0620 23:08:27.763257 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:27.763259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:27.763267 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:27.763309 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:27.763310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:27.763319 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:27.763372 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:27.763891 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.764420 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.765138 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.765320 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.765366 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:27.767574 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:27.767576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:27.767576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:27.767582 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:27.769256 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:27.769562 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:27.769563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:27.769563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:27.769642 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:27.769820 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:27.769821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:27.769821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:27.769922 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:27.769997 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:27.770080 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:08:27.770142 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260620 23:08:27.771727 2875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:27.771890 2875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:27.773253 3013 tablet_service.cc:1467] Processing CreateTablet for tablet 02a9c5d846804f2ab7421e840dd854f5 (DEFAULT_TABLE table=test-table [id=730fba0ab8fb43a39c063c467f0f07c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:27.773257 3153 tablet_service.cc:1467] Processing CreateTablet for tablet 02a9c5d846804f2ab7421e840dd854f5 (DEFAULT_TABLE table=test-table [id=730fba0ab8fb43a39c063c467f0f07c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:27.773269 3293 tablet_service.cc:1467] Processing CreateTablet for tablet 02a9c5d846804f2ab7421e840dd854f5 (DEFAULT_TABLE table=test-table [id=730fba0ab8fb43a39c063c467f0f07c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:27.773386 3013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a9c5d846804f2ab7421e840dd854f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:27.773388 3293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a9c5d846804f2ab7421e840dd854f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:27.773388 3153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a9c5d846804f2ab7421e840dd854f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:27.774307 3459 tablet_bootstrap.cc:492] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: Bootstrap starting.
I20260620 23:08:27.774621 3459 tablet_bootstrap.cc:654] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:27.774827 3461 tablet_bootstrap.cc:492] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: Bootstrap starting.
I20260620 23:08:27.775010 3460 tablet_bootstrap.cc:492] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Bootstrap starting.
I20260620 23:08:27.775220 3461 tablet_bootstrap.cc:654] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:27.775257 3459 tablet_bootstrap.cc:492] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: No bootstrap required, opened a new log
I20260620 23:08:27.775295 3459 ts_tablet_manager.cc:1397] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:27.775434 3459 raft_consensus.cc:348] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.775453 3460 tablet_bootstrap.cc:654] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:27.775503 3459 raft_consensus.cc:374] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:27.775544 3459 raft_consensus.cc:729] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Initialized, Role: FOLLOWER
I20260620 23:08:27.775607 3459 consensus_queue.cc:260] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.775700 3459 ts_tablet_manager.cc:1428] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:27.775836 3461 tablet_bootstrap.cc:492] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: No bootstrap required, opened a new log
I20260620 23:08:27.775913 3461 ts_tablet_manager.cc:1397] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:27.775998 3460 tablet_bootstrap.cc:492] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: No bootstrap required, opened a new log
I20260620 23:08:27.776052 3460 ts_tablet_manager.cc:1397] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:27.776026 3461 raft_consensus.cc:348] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.776084 3461 raft_consensus.cc:374] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:27.776103 3461 raft_consensus.cc:729] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99abacd5b944ff0b78a2695b4cf7527, State: Initialized, Role: FOLLOWER
I20260620 23:08:27.776142 3461 consensus_queue.cc:260] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.776170 3460 raft_consensus.cc:348] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.776232 3460 raft_consensus.cc:374] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:27.776266 3460 raft_consensus.cc:729] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51a3de1cbbc48f6a63c1fedaf4ced6e, State: Initialized, Role: FOLLOWER
I20260620 23:08:27.776283 3461 ts_tablet_manager.cc:1428] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:27.776311 3460 consensus_queue.cc:260] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.776468 3460 ts_tablet_manager.cc:1428] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:27.988742 3465 raft_consensus.cc:482] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:27.988857 3465 raft_consensus.cc:504] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.989238 3465 leader_election.cc:288] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65:45537), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:27.992213 3468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:27.989461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39213 (local address 127.1.32.66:36105)
0620 23:08:27.989789 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:27.989801 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:27.989882 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:27.990037 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:27.990037 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:27.990039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:27.990106 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:27.990107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.990763 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.990763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:27.991467 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:27.991469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.991961 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.991962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:27.992019 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:27.992022 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:27.992083 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:08:27.992109 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":2,"thread_start_us":128,"threads_started":1}
I20260620 23:08:27.992221 3469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:27.989570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:36105 (local address 127.1.32.67:39213)
0620 23:08:27.989721 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:27.989724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:27.989904 (+ 180us) server_negotiation.cc:407] Connection header received
0620 23:08:27.989935 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:27.989936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:27.989946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:27.990023 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:27.990181 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.990674 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.991552 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.991885 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.991935 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:27.991943 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:27.992095 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:08:27.992152 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260620 23:08:27.992286 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:27.989543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:36293 (local address 127.1.32.65:45537)
0620 23:08:27.989721 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:27.989724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:27.989773 (+ 49us) server_negotiation.cc:407] Connection header received
0620 23:08:27.989827 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:27.989828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:27.989838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:27.989912 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:27.990094 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.990615 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.991590 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.991948 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.992014 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:27.992025 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:27.992166 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 23:08:27.992211 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
I20260620 23:08:27.992324 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:27.989461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:45537 (local address 127.1.32.66:36293)
0620 23:08:27.989696 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:27.989707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:27.989772 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:27.989931 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:27.989932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:27.989934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:27.990010 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:27.990012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.990696 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.990697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:27.991474 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:27.991476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:27.992027 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:27.992027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:27.992093 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:27.992096 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:27.992148 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:08:27.992183 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":1,"spinlock_wait_cycles":35072,"thread_start_us":133,"threads_started":1}
I20260620 23:08:27.992683 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9c5d846804f2ab7421e840dd854f5" candidate_uuid: "f99abacd5b944ff0b78a2695b4cf7527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" is_pre_election: true
I20260620 23:08:27.992781 3312 raft_consensus.cc:2454] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99abacd5b944ff0b78a2695b4cf7527 in term 0.
I20260620 23:08:27.992734 3033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9c5d846804f2ab7421e840dd854f5" candidate_uuid: "f99abacd5b944ff0b78a2695b4cf7527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f0928cb434409aaddd4b7cbd575eb0" is_pre_election: true
I20260620 23:08:27.992877 3033 raft_consensus.cc:2454] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99abacd5b944ff0b78a2695b4cf7527 in term 0.
I20260620 23:08:27.992946 3106 leader_election.cc:302] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f51a3de1cbbc48f6a63c1fedaf4ced6e, f99abacd5b944ff0b78a2695b4cf7527; no voters:
I20260620 23:08:27.993083 3465 raft_consensus.cc:2781] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:27.993137 3465 raft_consensus.cc:482] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:27.993151 3465 raft_consensus.cc:3037] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:27.993738 3465 raft_consensus.cc:504] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.993862 3465 leader_election.cc:288] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [CANDIDATE]: Term 1 election: Requested vote from peers 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65:45537), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:27.994030 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9c5d846804f2ab7421e840dd854f5" candidate_uuid: "f99abacd5b944ff0b78a2695b4cf7527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
I20260620 23:08:27.994107 3312 raft_consensus.cc:3037] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:27.994656 3312 raft_consensus.cc:2454] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99abacd5b944ff0b78a2695b4cf7527 in term 1.
I20260620 23:08:27.994765 3033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9c5d846804f2ab7421e840dd854f5" candidate_uuid: "f99abacd5b944ff0b78a2695b4cf7527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f0928cb434409aaddd4b7cbd575eb0"
I20260620 23:08:27.994792 3106 leader_election.cc:302] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f51a3de1cbbc48f6a63c1fedaf4ced6e, f99abacd5b944ff0b78a2695b4cf7527; no voters:
I20260620 23:08:27.994841 3033 raft_consensus.cc:3037] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:27.994899 3465 raft_consensus.cc:2781] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:27.995023 3465 raft_consensus.cc:686] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 LEADER]: Becoming Leader. State: Replica: f99abacd5b944ff0b78a2695b4cf7527, State: Running, Role: LEADER
I20260620 23:08:27.995110 3465 consensus_queue.cc:237] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:27.995641 3033 raft_consensus.cc:2454] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99abacd5b944ff0b78a2695b4cf7527 in term 1.
I20260620 23:08:27.995826 2876 catalog_manager.cc:5495] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66). New cstate: current_term: 1 leader_uuid: "f99abacd5b944ff0b78a2695b4cf7527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:28.003985 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:28.001806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.1.32.66:45875)
0620 23:08:28.001926 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:28.001928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:28.001935 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:28.001973 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:28.001973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:28.002003 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:28.002064 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:28.002201 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.002618 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.003307 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.003489 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.003537 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:28.003709 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:28.003799 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:28.003862 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:28.003884 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 23:08:28.003900 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":79,"threads_started":1}
I20260620 23:08:28.008680 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:28.006320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:55469 (local address 127.1.32.65:45537)
0620 23:08:28.006387 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:28.006389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:28.006488 (+ 99us) server_negotiation.cc:407] Connection header received
0620 23:08:28.006549 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:28.006550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:28.006562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:28.006634 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:28.006784 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.007261 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.008001 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.008190 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.008270 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:28.008439 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:28.008469 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:28.008536 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:28.008586 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 23:08:28.008614 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:08:28.008698 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:28.006234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:45537 (local address 127.1.32.66:55469)
0620 23:08:28.006411 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:28.006422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:28.006497 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:28.006638 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:28.006639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:28.006640 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:28.006709 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:28.006710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.007351 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.007352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:28.007903 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:28.007904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.008273 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.008274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:28.008345 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:28.008357 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:28.008536 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:28.008538 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:08:28.008586 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:08:28.008623 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260620 23:08:28.010795 3312 raft_consensus.cc:1261] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Refusing update from remote peer f99abacd5b944ff0b78a2695b4cf7527: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:28.010933 3033 raft_consensus.cc:1261] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Refusing update from remote peer f99abacd5b944ff0b78a2695b4cf7527: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:28.011035 3465 consensus_queue.cc:1035] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:28.011152 3471 consensus_queue.cc:1035] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:28.017706 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:28.015719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:54889 (local address 127.1.32.66:45875)
0620 23:08:28.015751 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:28.015752 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:28.015822 (+ 70us) server_negotiation.cc:407] Connection header received
0620 23:08:28.015879 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:28.015880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:28.015889 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:28.015940 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:28.016088 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.016462 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.017052 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.017216 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.017266 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:28.017417 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:28.017445 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:28.017494 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:28.017636 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 23:08:28.017654 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:08:28.017732 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:28.015672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45875 (local address 127.1.32.65:54889)
0620 23:08:28.015773 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:28.015782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:28.015832 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:28.015997 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:28.015998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:28.015999 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:28.016035 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:28.016036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.016516 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.016517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:28.016987 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:28.016988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.017276 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.017277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:28.017336 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:28.017340 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:28.017563 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:28.017564 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:08:28.017629 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:08:28.017667 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":69,"threads_started":1}
I20260620 23:08:28.024276 2876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:28.024377 2876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:28.024972 2876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Sending DeleteTablet for 3 replicas of tablet 02a9c5d846804f2ab7421e840dd854f5
I20260620 23:08:28.025220 3153 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a9c5d846804f2ab7421e840dd854f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:28 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39376
I20260620 23:08:28.025221 3293 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a9c5d846804f2ab7421e840dd854f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:28 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260620 23:08:28.025275 3013 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a9c5d846804f2ab7421e840dd854f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:28 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:50168
I20260620 23:08:28.025297 3449 tablet_replica.cc:330] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: stopping tablet replica
I20260620 23:08:28.025297 3450 tablet_replica.cc:330] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: stopping tablet replica
I20260620 23:08:28.025337 3450 raft_consensus.cc:2229] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:28.025337 3449 raft_consensus.cc:2229] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:28.025389 3450 raft_consensus.cc:2258] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:28.025439 3449 raft_consensus.cc:2258] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:28.025461 3448 tablet_replica.cc:330] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: stopping tablet replica
I20260620 23:08:28.025514 3448 raft_consensus.cc:2229] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:28.025547 3448 raft_consensus.cc:2258] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:28.025868 3449 ts_tablet_manager.cc:1905] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:28.025986 3448 ts_tablet_manager.cc:1905] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:28.027511 3449 ts_tablet_manager.cc:1918] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:28.027567 3449 log.cc:1196] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/02a9c5d846804f2ab7421e840dd854f5
I20260620 23:08:28.027559 3448 ts_tablet_manager.cc:1918] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:28.027581 3450 ts_tablet_manager.cc:1905] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:28.027626 3448 log.cc:1196] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/02a9c5d846804f2ab7421e840dd854f5
I20260620 23:08:28.027798 3449 ts_tablet_manager.cc:1939] T 02a9c5d846804f2ab7421e840dd854f5 P f99abacd5b944ff0b78a2695b4cf7527: Deleting consensus metadata
I20260620 23:08:28.027884 3448 ts_tablet_manager.cc:1939] T 02a9c5d846804f2ab7421e840dd854f5 P 15f0928cb434409aaddd4b7cbd575eb0: Deleting consensus metadata
I20260620 23:08:28.028055 2860 catalog_manager.cc:4841] TS f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875): tablet 02a9c5d846804f2ab7421e840dd854f5 (table test-table [id=730fba0ab8fb43a39c063c467f0f07c3]) successfully deleted
I20260620 23:08:28.028179 2861 catalog_manager.cc:4841] TS 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65:45537): tablet 02a9c5d846804f2ab7421e840dd854f5 (table test-table [id=730fba0ab8fb43a39c063c467f0f07c3]) successfully deleted
I20260620 23:08:28.028808 3450 ts_tablet_manager.cc:1918] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:28.028852 3450 log.cc:1196] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/02a9c5d846804f2ab7421e840dd854f5
I20260620 23:08:28.029053 3450 ts_tablet_manager.cc:1939] T 02a9c5d846804f2ab7421e840dd854f5 P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting consensus metadata
I20260620 23:08:28.029350 2862 catalog_manager.cc:4841] TS f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213): tablet 02a9c5d846804f2ab7421e840dd854f5 (table test-table [id=730fba0ab8fb43a39c063c467f0f07c3]) successfully deleted
I20260620 23:08:28.029691 2876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:28.029840 2876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:28.031001 3013 tablet_service.cc:1467] Processing CreateTablet for tablet 6a4730b4ebb541d895d36cf49f277fee (DEFAULT_TABLE table=test-table [id=7a3502b7b3d44e0e8ef300a8042d5dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:28.031044 3293 tablet_service.cc:1467] Processing CreateTablet for tablet 6a4730b4ebb541d895d36cf49f277fee (DEFAULT_TABLE table=test-table [id=7a3502b7b3d44e0e8ef300a8042d5dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:28.031113 3013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a4730b4ebb541d895d36cf49f277fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:28.031117 3293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a4730b4ebb541d895d36cf49f277fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:28.031775 3153 tablet_service.cc:1467] Processing CreateTablet for tablet 6a4730b4ebb541d895d36cf49f277fee (DEFAULT_TABLE table=test-table [id=7a3502b7b3d44e0e8ef300a8042d5dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:28.031873 3153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a4730b4ebb541d895d36cf49f277fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:28.032315 3459 tablet_bootstrap.cc:492] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0: Bootstrap starting.
I20260620 23:08:28.032593 3459 tablet_bootstrap.cc:654] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:28.033080 3461 tablet_bootstrap.cc:492] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527: Bootstrap starting.
I20260620 23:08:28.033165 3459 tablet_bootstrap.cc:492] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0: No bootstrap required, opened a new log
I20260620 23:08:28.033210 3459 ts_tablet_manager.cc:1397] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:08:28.033310 3459 raft_consensus.cc:348] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.033365 3459 raft_consensus.cc:374] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:28.033382 3459 raft_consensus.cc:729] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Initialized, Role: FOLLOWER
I20260620 23:08:28.033420 3459 consensus_queue.cc:260] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.033471 3461 tablet_bootstrap.cc:654] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:28.033495 3459 ts_tablet_manager.cc:1428] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:28.033840 3460 tablet_bootstrap.cc:492] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: Bootstrap starting.
I20260620 23:08:28.034154 3460 tablet_bootstrap.cc:654] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:28.034353 3461 tablet_bootstrap.cc:492] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527: No bootstrap required, opened a new log
I20260620 23:08:28.034397 3461 ts_tablet_manager.cc:1397] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:28.034533 3461 raft_consensus.cc:348] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.034626 3461 raft_consensus.cc:374] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:28.034654 3461 raft_consensus.cc:729] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99abacd5b944ff0b78a2695b4cf7527, State: Initialized, Role: FOLLOWER
I20260620 23:08:28.034693 3461 consensus_queue.cc:260] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.034754 3460 tablet_bootstrap.cc:492] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: No bootstrap required, opened a new log
I20260620 23:08:28.034806 3460 ts_tablet_manager.cc:1397] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:28.034811 3461 ts_tablet_manager.cc:1428] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:28.034967 3460 raft_consensus.cc:348] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.035036 3460 raft_consensus.cc:374] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:28.035059 3460 raft_consensus.cc:729] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51a3de1cbbc48f6a63c1fedaf4ced6e, State: Initialized, Role: FOLLOWER
I20260620 23:08:28.035111 3460 consensus_queue.cc:260] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.035195 3460 ts_tablet_manager.cc:1428] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:28.273176 3431 raft_consensus.cc:482] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:28.273273 3431 raft_consensus.cc:504] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.273456 3431 leader_election.cc:288] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:28.273685 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4730b4ebb541d895d36cf49f277fee" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" is_pre_election: true
I20260620 23:08:28.273694 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4730b4ebb541d895d36cf49f277fee" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527" is_pre_election: true
I20260620 23:08:28.273792 3173 raft_consensus.cc:2454] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:28.273794 3312 raft_consensus.cc:2454] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 0.
I20260620 23:08:28.273978 2966 leader_election.cc:302] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f51a3de1cbbc48f6a63c1fedaf4ced6e; no voters:
I20260620 23:08:28.274067 3431 raft_consensus.cc:2781] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:28.274137 3431 raft_consensus.cc:482] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:28.274166 3431 raft_consensus.cc:3037] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:28.274866 3431 raft_consensus.cc:504] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.275074 3431 leader_election.cc:288] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Requested vote from peers f99abacd5b944ff0b78a2695b4cf7527 (127.1.32.66:45875), f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213)
I20260620 23:08:28.275219 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4730b4ebb541d895d36cf49f277fee" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e"
I20260620 23:08:28.275226 3173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4730b4ebb541d895d36cf49f277fee" candidate_uuid: "15f0928cb434409aaddd4b7cbd575eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99abacd5b944ff0b78a2695b4cf7527"
I20260620 23:08:28.275290 3173 raft_consensus.cc:3037] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:28.275306 3312 raft_consensus.cc:3037] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:28.275872 3312 raft_consensus.cc:2454] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:28.275873 3173 raft_consensus.cc:2454] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f0928cb434409aaddd4b7cbd575eb0 in term 1.
I20260620 23:08:28.276028 2966 leader_election.cc:302] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f0928cb434409aaddd4b7cbd575eb0, f51a3de1cbbc48f6a63c1fedaf4ced6e; no voters:
I20260620 23:08:28.276118 3431 raft_consensus.cc:2781] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:28.276248 3431 raft_consensus.cc:686] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [term 1 LEADER]: Becoming Leader. State: Replica: 15f0928cb434409aaddd4b7cbd575eb0, State: Running, Role: LEADER
I20260620 23:08:28.276301 3431 consensus_queue.cc:237] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } }
I20260620 23:08:28.276880 2874 catalog_manager.cc:5495] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65). New cstate: current_term: 1 leader_uuid: "15f0928cb434409aaddd4b7cbd575eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f0928cb434409aaddd4b7cbd575eb0" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 45537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:28.319448 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:28.317339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.1.32.65:45537)
0620 23:08:28.317377 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:28.317378 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:28.317413 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:08:28.317494 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:28.317494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:28.317502 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:28.317540 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:28.317670 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.318109 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.318765 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:28.319004 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:28.319053 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:28.319212 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:28.319241 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:28.319312 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:28.319368 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 23:08:28.319393 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:28.321487 3173 raft_consensus.cc:1261] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:28.321548 3312 raft_consensus.cc:1261] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Refusing update from remote peer 15f0928cb434409aaddd4b7cbd575eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:28.321707 3436 consensus_queue.cc:1035] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99abacd5b944ff0b78a2695b4cf7527" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:28.321861 3411 consensus_queue.cc:1035] T 6a4730b4ebb541d895d36cf49f277fee P 15f0928cb434409aaddd4b7cbd575eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51a3de1cbbc48f6a63c1fedaf4ced6e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:08:28.326123 3429 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 23:08:29.842996 2876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:29.843088 2876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:29.843988 2876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a93daf67be74b85b4027afdf4ca2213: Sending DeleteTablet for 3 replicas of tablet 6a4730b4ebb541d895d36cf49f277fee
I20260620 23:08:29.844220 3153 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a4730b4ebb541d895d36cf49f277fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:29 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39376
I20260620 23:08:29.844226 3293 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a4730b4ebb541d895d36cf49f277fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:29 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260620 23:08:29.844264 3013 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a4730b4ebb541d895d36cf49f277fee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:29 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:50168
I20260620 23:08:29.844399 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 2948
I20260620 23:08:29.851047 3510 tablet_replica.cc:330] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527: stopping tablet replica
W20260620 23:08:29.851066 2861 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:45537 (error 108)
I20260620 23:08:29.851075 3511 tablet_replica.cc:330] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: stopping tablet replica
I20260620 23:08:29.851140 3511 raft_consensus.cc:2229] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:29.851162 3510 raft_consensus.cc:2229] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:29.851197 3511 raft_consensus.cc:2258] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:29.851217 3510 raft_consensus.cc:2258] T 6a4730b4ebb541d895d36cf49f277fee P f99abacd5b944ff0b78a2695b4cf7527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:29.851229 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 3088
I20260620 23:08:29.851470 3511 ts_tablet_manager.cc:1905] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:29.852985 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:29.851941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:45537 (local address 127.0.0.1:60560)
0620 23:08:29.852898 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:29.852944 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":51,"threads_started":1}
W20260620 23:08:29.853091 2861 catalog_manager.cc:4584] TS 15f0928cb434409aaddd4b7cbd575eb0 (127.1.32.65:45537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a4730b4ebb541d895d36cf49f277fee: Network error: Client connection negotiation failed: client connection to 127.1.32.65:45537: connect: Connection refused (error 111)
I20260620 23:08:29.853358 3511 ts_tablet_manager.cc:1918] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 23:08:29.853422 3511 log.cc:1196] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/6a4730b4ebb541d895d36cf49f277fee
I20260620 23:08:29.853632 3511 ts_tablet_manager.cc:1939] T 6a4730b4ebb541d895d36cf49f277fee P f51a3de1cbbc48f6a63c1fedaf4ced6e: Deleting consensus metadata
I20260620 23:08:29.854115 2862 catalog_manager.cc:4841] TS f51a3de1cbbc48f6a63c1fedaf4ced6e (127.1.32.67:39213): tablet 6a4730b4ebb541d895d36cf49f277fee (table test-table [id=7a3502b7b3d44e0e8ef300a8042d5dde]) successfully deleted
I20260620 23:08:29.856827 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:29.856714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45875 (local address 127.0.0.1:39006)
0620 23:08:29.856766 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:29.856791 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 23:08:29.856825 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 3228
I20260620 23:08:29.861411 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 2844
2026-06-20T23:08:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5719 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:29 dist-test-slave-fzjm krb5kdc[3517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:29 dist-test-slave-fzjm krb5kdc[3517](info): set up 2 sockets
Jun 20 23:08:29 dist-test-slave-fzjm krb5kdc[3517](info): commencing operation
krb5kdc: starting...
W20260620 23:08:30.850325 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:30 dist-test-slave-fzjm krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:30.955583 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40931
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:40931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:31.030124 3530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:31.030299 3530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:31.030320 3530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:31.031301 3530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:31.031338 3530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:31.031352 3530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:31.031366 3530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:31.031378 3530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:31.032603 3530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:40931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:31.032766 3530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:31.032915 3530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:31.037513 3536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:31.037616 3538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:31.037693 3530 server_base.cc:1034] running on GCE node
W20260620 23:08:31.037521 3535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:31.037918 3530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:31.038302 3530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:31.039453 3530 hybrid_clock.cc:648] HybridClock initialized: now 1781996911039433 us; error 42 us; skew 500 ppm
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996911, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:31.041661 3530 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:31.042101 3530 webserver.cc:460] Webserver started at http://127.1.32.126:44617/ using document root <none> and password file <none>
I20260620 23:08:31.042313 3530 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:31.042377 3530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:31.042522 3530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:31.043598 3530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "48a23345c0ac42e18e270eb1885d31e8"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "dfb34a8a8fe2e294d733bdc998f4cfc7"
server_key_iv: "a21fc9b1a58cf4807bddb5c7dbf5c2ba"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.043952 3530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "48a23345c0ac42e18e270eb1885d31e8"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "dfb34a8a8fe2e294d733bdc998f4cfc7"
server_key_iv: "a21fc9b1a58cf4807bddb5c7dbf5c2ba"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.045500 3530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:08:31.046339 3545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.046515 3530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:31.046644 3530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "48a23345c0ac42e18e270eb1885d31e8"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "dfb34a8a8fe2e294d733bdc998f4cfc7"
server_key_iv: "a21fc9b1a58cf4807bddb5c7dbf5c2ba"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.046720 3530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:31.067682 3530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:31.069283 3530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:31.069432 3530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:31.073027 3530 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:40931
I20260620 23:08:31.073060 3597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:40931 every 8 connection(s)
I20260620 23:08:31.073360 3530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:31.074102 3598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:31.076288 3598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8: Bootstrap starting.
I20260620 23:08:31.076824 3598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:31.077101 3598 log.cc:824] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:31.077708 3598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8: No bootstrap required, opened a new log
I20260620 23:08:31.079094 3598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a23345c0ac42e18e270eb1885d31e8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40931 } }
I20260620 23:08:31.079208 3598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:31.079228 3598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a23345c0ac42e18e270eb1885d31e8, State: Initialized, Role: FOLLOWER
I20260620 23:08:31.079357 3598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a23345c0ac42e18e270eb1885d31e8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40931 } }
I20260620 23:08:31.079447 3598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:31.079479 3598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:31.079527 3598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:31.079574 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 3530
I20260620 23:08:31.079665 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:31.079854 1153 external_mini_cluster.cc:1428] Setting key f59960a0a5c8c8befd1997e3b2dee5ed
I20260620 23:08:31.080119 3598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a23345c0ac42e18e270eb1885d31e8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40931 } }
I20260620 23:08:31.080230 3598 leader_election.cc:302] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48a23345c0ac42e18e270eb1885d31e8; no voters:
I20260620 23:08:31.080404 3598 leader_election.cc:288] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:31.080633 3598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:31.080747 3604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:31.080892 3604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [term 1 LEADER]: Becoming Leader. State: Replica: 48a23345c0ac42e18e270eb1885d31e8, State: Running, Role: LEADER
I20260620 23:08:31.081056 3604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a23345c0ac42e18e270eb1885d31e8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40931 } }
I20260620 23:08:31.081501 3605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48a23345c0ac42e18e270eb1885d31e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a23345c0ac42e18e270eb1885d31e8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40931 } } }
I20260620 23:08:31.081601 3605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:31.081737 3605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48a23345c0ac42e18e270eb1885d31e8. Latest consensus state: current_term: 1 leader_uuid: "48a23345c0ac42e18e270eb1885d31e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a23345c0ac42e18e270eb1885d31e8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40931 } } }
I20260620 23:08:31.081789 3605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:31.081818 3614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:31.082402 3614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:31.083971 3614 catalog_manager.cc:1347] Generated new cluster ID: 07cdf7b240ef4aa38a66342d96d91783
I20260620 23:08:31.084023 3614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:31.089696 3602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:31.080284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.1.32.126:40931)
0620 23:08:31.080472 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:31.080480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:31.080514 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:08:31.080630 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:31.080638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:31.080725 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:31.081712 (+ 987us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:31.081818 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.083131 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.083993 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.084464 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.084586 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:31.086705 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:31.086708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:31.086708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:31.086719 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:31.088347 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.088694 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.088696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.088697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.088791 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.088982 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.088983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.088983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.089198 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:31.089350 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:31.089406 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 23:08:31.089523 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
I20260620 23:08:31.097430 3614 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:31.098078 3614 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:31.103358 3614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48a23345c0ac42e18e270eb1885d31e8: Generated new TSK 0
I20260620 23:08:31.103498 3614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:31.135214 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40931
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:31.211261 3624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:31.211438 3624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:31.211458 3624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:31.212507 3624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:31.212545 3624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:31.212568 3624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:31.213970 3624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:31.214206 3624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:31.214449 3624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:31.219228 3629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:31.219209 3632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:31.219203 3630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:31.219419 3624 server_base.cc:1034] running on GCE node
I20260620 23:08:31.219645 3624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:31.219877 3624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:31.221038 3624 hybrid_clock.cc:648] HybridClock initialized: now 1781996911221016 us; error 36 us; skew 500 ppm
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996911, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:31.223472 3624 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:31.223884 3624 webserver.cc:460] Webserver started at http://127.1.32.65:33943/ using document root <none> and password file <none>
I20260620 23:08:31.224174 3624 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:31.224241 3624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:31.224422 3624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:31.225500 3624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "0bcef6ad53284d768c029fbb303534d4"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "27899ad5b7cbd5222e629f883075f0a9"
server_key_iv: "09b1442ff12336fbf70e86eb6ddb2b86"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.225844 3624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "0bcef6ad53284d768c029fbb303534d4"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "27899ad5b7cbd5222e629f883075f0a9"
server_key_iv: "09b1442ff12336fbf70e86eb6ddb2b86"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.227190 3624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:08:31.227857 3639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.228008 3624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:31.228096 3624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "0bcef6ad53284d768c029fbb303534d4"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "27899ad5b7cbd5222e629f883075f0a9"
server_key_iv: "09b1442ff12336fbf70e86eb6ddb2b86"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.228169 3624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:31.235821 3624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:31.237810 3624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:31.237950 3624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:31.238137 3624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:31.238401 3624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:31.238449 3624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.238483 3624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:31.238508 3624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.244027 3624 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:45211
I20260620 23:08:31.244087 3752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:45211 every 8 connection(s)
I20260620 23:08:31.244408 3624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:31.249233 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 3624
I20260620 23:08:31.249328 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:31.249506 1153 external_mini_cluster.cc:1428] Setting key 0da3b0ff9de1ff080448b5a21a5fda83
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996911, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:31.253298 3602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:31.245095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:36647 (local address 127.1.32.126:40931)
0620 23:08:31.245157 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:31.245159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:31.245397 (+ 238us) server_negotiation.cc:407] Connection header received
0620 23:08:31.245809 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:31.245810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:31.245822 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:31.245868 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:31.246639 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.247168 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.248022 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.248195 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.248239 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:31.250349 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:31.250351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:31.250351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:31.250360 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:31.251815 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.252293 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.252294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.252294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.252344 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.252628 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.252629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.252629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.252765 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:31.252865 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:31.253129 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 23:08:31.253210 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:08:31.253484 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:31.245080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40931 (local address 127.1.32.65:36647)
0620 23:08:31.245273 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:31.245308 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:31.245737 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:31.245913 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:31.245915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:31.246385 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:31.246544 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:31.246547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.247238 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.247239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:31.247958 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:31.247960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.248262 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.248263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:31.248359 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:31.248718 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:31.248725 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:31.250267 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:31.251893 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:31.251900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:31.251902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:31.252215 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:31.252417 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:31.252418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:31.252419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:31.252522 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:31.252857 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:31.252864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:31.252994 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 23:08:31.253107 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 23:08:31.253253 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260620 23:08:31.253917 3753 heartbeater.cc:347] Connected to a master server at 127.1.32.126:40931
I20260620 23:08:31.254032 3753 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:31.254220 3753 heartbeater.cc:511] Master 127.1.32.126:40931 requested a full tablet report, sending...
I20260620 23:08:31.254720 3562 ts_manager.cc:194] Registered new tserver with Master: 0bcef6ad53284d768c029fbb303534d4 (127.1.32.65:45211)
I20260620 23:08:31.255359 3562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:36647
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:31.272712 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40931
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:31.347414 3758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:31.347586 3758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:31.347606 3758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:31.348558 3758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:31.348595 3758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:31.348619 3758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:31.349817 3758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:31.349958 3758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:31.350119 3758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:31.354868 3763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:31.354928 3764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:31.355002 3766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:31.354954 3758 server_base.cc:1034] running on GCE node
I20260620 23:08:31.355404 3758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:31.355643 3758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:31.356796 3758 hybrid_clock.cc:648] HybridClock initialized: now 1781996911356783 us; error 28 us; skew 500 ppm
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996911, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:31.359143 3758 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:31.359555 3758 webserver.cc:460] Webserver started at http://127.1.32.66:34403/ using document root <none> and password file <none>
I20260620 23:08:31.359728 3758 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:31.359764 3758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:31.359843 3758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:31.360677 3758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "d81b11b2c821453288e3c9bc2aee1dfc"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "024a5d3a1dee4231988782cdd241528a"
server_key_iv: "cd9a915c64cf79272ca9124cffdba996"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.360931 3758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "d81b11b2c821453288e3c9bc2aee1dfc"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "024a5d3a1dee4231988782cdd241528a"
server_key_iv: "cd9a915c64cf79272ca9124cffdba996"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.362162 3758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:31.362799 3773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.362967 3758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:31.363047 3758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "d81b11b2c821453288e3c9bc2aee1dfc"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "024a5d3a1dee4231988782cdd241528a"
server_key_iv: "cd9a915c64cf79272ca9124cffdba996"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.363109 3758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:31.379277 3758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:31.381260 3758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:31.381366 3758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:31.381572 3758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:31.381857 3758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:31.381887 3758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.381908 3758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:31.381920 3758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.387527 3758 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:45823
I20260620 23:08:31.387630 3886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:45823 every 8 connection(s)
I20260620 23:08:31.387894 3758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996911, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:31.396546 3602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:31.388593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:37641 (local address 127.1.32.126:40931)
0620 23:08:31.388674 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:31.388677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:31.388849 (+ 172us) server_negotiation.cc:407] Connection header received
0620 23:08:31.389355 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:31.389356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:31.389368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:31.389419 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:31.390348 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.390851 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.391873 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.392077 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.392149 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:31.394083 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:31.394084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:31.394085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:31.394091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:31.395295 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.395708 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.395709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.395710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.395764 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.395978 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.395979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.395980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.396088 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:31.396157 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:31.396389 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 23:08:31.396465 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 23:08:31.396621 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 3758
I20260620 23:08:31.396714 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:31.396729 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:31.388563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40931 (local address 127.1.32.66:37641)
0620 23:08:31.388725 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:31.388770 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:31.389252 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:31.389488 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:31.389492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:31.390066 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:31.390244 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:31.390248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.390982 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.390983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:31.391772 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:31.391774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.392153 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.392154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:31.392280 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:31.392777 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:31.392785 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:31.393986 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:31.395407 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:31.395412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:31.395414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:31.395624 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:31.395823 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:31.395823 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:31.395824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:31.395916 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:31.396147 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:31.396153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:31.396275 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 23:08:31.396371 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:08:31.396508 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260620 23:08:31.396888 1153 external_mini_cluster.cc:1428] Setting key 2860771037c4681bb2ada8e7f86b78a0
I20260620 23:08:31.397161 3887 heartbeater.cc:347] Connected to a master server at 127.1.32.126:40931
I20260620 23:08:31.397241 3887 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:31.397394 3887 heartbeater.cc:511] Master 127.1.32.126:40931 requested a full tablet report, sending...
I20260620 23:08:31.397751 3562 ts_manager.cc:194] Registered new tserver with Master: d81b11b2c821453288e3c9bc2aee1dfc (127.1.32.66:45823)
I20260620 23:08:31.398198 3562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:37641
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:31.419618 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40931
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:31.498064 3892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:31.498298 3892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:31.498337 3892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:31.499502 3892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:31.499583 3892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:31.499631 3892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:31.501009 3892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:31.501263 3892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:31.501497 3892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:31.506331 3897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:31.506337 3898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:31.506403 3900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:31.506363 3892 server_base.cc:1034] running on GCE node
I20260620 23:08:31.506929 3892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:31.507205 3892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:31.508353 3892 hybrid_clock.cc:648] HybridClock initialized: now 1781996911508344 us; error 33 us; skew 500 ppm
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996911, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:31.510721 3892 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:31.511253 3892 webserver.cc:460] Webserver started at http://127.1.32.67:39639/ using document root <none> and password file <none>
I20260620 23:08:31.511474 3892 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:31.511540 3892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:31.511691 3892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:31.512727 3892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "10b17936cc034eb78dcc50969999d8ac"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "555e2204923eb6de9c8d1ab416a2b5b6"
server_key_iv: "bdc471a75dfcafd726f26373ae1697ea"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.513099 3892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "10b17936cc034eb78dcc50969999d8ac"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "555e2204923eb6de9c8d1ab416a2b5b6"
server_key_iv: "bdc471a75dfcafd726f26373ae1697ea"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.514425 3892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:31.515172 3907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.515344 3892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:31.515441 3892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "10b17936cc034eb78dcc50969999d8ac"
format_stamp: "Formatted at 2026-06-20 23:08:31 on dist-test-slave-fzjm"
server_key: "555e2204923eb6de9c8d1ab416a2b5b6"
server_key_iv: "bdc471a75dfcafd726f26373ae1697ea"
server_key_version: "encryptionkey@0"
I20260620 23:08:31.515520 3892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:31.526755 3892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:31.528623 3892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:31.528788 3892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:31.529009 3892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:31.529307 3892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:31.529361 3892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.529399 3892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:31.529429 3892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:31.535429 3892 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:44981
I20260620 23:08:31.535461 4020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:44981 every 8 connection(s)
I20260620 23:08:31.535786 3892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996911, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:31.543659 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 3892
I20260620 23:08:31.543787 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:31.544005 1153 external_mini_cluster.cc:1428] Setting key 7f74082eb8149cf4b6a7309e3c889f9c
I20260620 23:08:31.544778 3602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:31.536495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:56693 (local address 127.1.32.126:40931)
0620 23:08:31.536573 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:31.536575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:31.536749 (+ 174us) server_negotiation.cc:407] Connection header received
0620 23:08:31.537208 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:31.537209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:31.537219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:31.537282 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:31.538193 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.538684 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.539755 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.539945 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.540000 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:31.542025 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:31.542026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:31.542026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:31.542033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:31.543271 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.543712 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.543713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.543713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.543790 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:31.544081 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:31.544082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:31.544082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:31.544217 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:31.544294 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:31.544615 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 23:08:31.544695 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:08:31.545044 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:31.536456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40931 (local address 127.1.32.67:56693)
0620 23:08:31.536616 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:31.536654 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:31.537113 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:31.537353 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:31.537357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:31.537917 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:31.538101 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:31.538105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.538759 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.538760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:31.539682 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:31.539685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:31.540015 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:31.540016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:31.540125 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:31.540629 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:31.540637 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:31.541943 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:31.543338 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:31.543344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:31.543345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:31.543624 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:31.543874 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:31.543874 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:31.543875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:31.544007 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:31.544299 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:31.544316 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:31.544471 (+ 155us) client_negotiation.cc:769] Sending connection context
0620 23:08:31.544581 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 23:08:31.544748 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260620 23:08:31.545466 4021 heartbeater.cc:347] Connected to a master server at 127.1.32.126:40931
I20260620 23:08:31.545560 4021 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:31.545696 4021 heartbeater.cc:511] Master 127.1.32.126:40931 requested a full tablet report, sending...
I20260620 23:08:31.546003 3562 ts_manager.cc:194] Registered new tserver with Master: 10b17936cc034eb78dcc50969999d8ac (127.1.32.67:44981)
I20260620 23:08:31.546411 3562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:56693
I20260620 23:08:31.555663 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:08:31.566162 3602 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:31.565303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.1.32.126:40931)
0620 23:08:31.565377 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:31.565379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:31.565487 (+ 108us) server_negotiation.cc:407] Connection header received
0620 23:08:31.565559 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:31.565560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:31.565574 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:31.565651 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:31.566106 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37592: BlockingRecv error: recv got EOF from 127.0.0.1:37592 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 23:08:31.567258 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 3624
I20260620 23:08:31.571866 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 3758
I20260620 23:08:31.575860 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 3892
I20260620 23:08:31.579597 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 3530
2026-06-20T23:08:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1716 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[4033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[4033](info): set up 2 sockets
Jun 20 23:08:31 dist-test-slave-fzjm krb5kdc[4033](info): commencing operation
krb5kdc: starting...
W20260620 23:08:32.558590 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:32 dist-test-slave-fzjm krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:32.671937 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36315
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:36315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:32.745519 4046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:32.745692 4046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:32.745713 4046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:32.746838 4046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:32.746876 4046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:32.746905 4046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:32.746917 4046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:32.746929 4046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:32.748317 4046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:36315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:32.748525 4046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:32.748726 4046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:32.753027 4052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:32.753036 4054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:32.753108 4051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:32.753201 4046 server_base.cc:1034] running on GCE node
I20260620 23:08:32.753428 4046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:32.753638 4046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:32.754771 4046 hybrid_clock.cc:648] HybridClock initialized: now 1781996912754739 us; error 47 us; skew 500 ppm
Jun 20 23:08:32 dist-test-slave-fzjm krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996912, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:32.757072 4046 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:32.757468 4046 webserver.cc:460] Webserver started at http://127.1.32.126:39427/ using document root <none> and password file <none>
I20260620 23:08:32.757680 4046 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:32.757742 4046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:32.757879 4046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:32.758823 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "d01730b6c042474091d386537d1c8a2a"
format_stamp: "Formatted at 2026-06-20 23:08:32 on dist-test-slave-fzjm"
server_key: "5c12c3404589a7e4a90d4cb6e6086403"
server_key_iv: "77979d93df27b5bff93160b23a0b5991"
server_key_version: "encryptionkey@0"
I20260620 23:08:32.759205 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "d01730b6c042474091d386537d1c8a2a"
format_stamp: "Formatted at 2026-06-20 23:08:32 on dist-test-slave-fzjm"
server_key: "5c12c3404589a7e4a90d4cb6e6086403"
server_key_iv: "77979d93df27b5bff93160b23a0b5991"
server_key_version: "encryptionkey@0"
I20260620 23:08:32.760366 4046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:32.760993 4061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:32.761173 4046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:32.761260 4046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "d01730b6c042474091d386537d1c8a2a"
format_stamp: "Formatted at 2026-06-20 23:08:32 on dist-test-slave-fzjm"
server_key: "5c12c3404589a7e4a90d4cb6e6086403"
server_key_iv: "77979d93df27b5bff93160b23a0b5991"
server_key_version: "encryptionkey@0"
I20260620 23:08:32.761353 4046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:32.775791 4046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:32.777951 4046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:32.778102 4046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:32.781958 4046 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:36315
I20260620 23:08:32.781992 4113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:36315 every 8 connection(s)
I20260620 23:08:32.782353 4046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:32.783164 4114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:32.785251 4114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a: Bootstrap starting.
I20260620 23:08:32.785770 4114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:32.785995 4114 log.cc:824] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:32.786377 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4046
I20260620 23:08:32.786470 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:32.786674 1153 external_mini_cluster.cc:1428] Setting key 7638e96a6fa38dce8327669ccc224e29
I20260620 23:08:32.786796 4114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a: No bootstrap required, opened a new log
I20260620 23:08:32.789573 4114 raft_consensus.cc:348] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01730b6c042474091d386537d1c8a2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36315 } }
I20260620 23:08:32.789736 4114 raft_consensus.cc:374] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:32.789767 4114 raft_consensus.cc:729] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d01730b6c042474091d386537d1c8a2a, State: Initialized, Role: FOLLOWER
I20260620 23:08:32.789875 4114 consensus_queue.cc:260] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01730b6c042474091d386537d1c8a2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36315 } }
I20260620 23:08:32.789940 4114 raft_consensus.cc:388] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:32.789968 4114 raft_consensus.cc:482] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:32.790011 4114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:32.790648 4114 raft_consensus.cc:504] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01730b6c042474091d386537d1c8a2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36315 } }
I20260620 23:08:32.790730 4114 leader_election.cc:302] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d01730b6c042474091d386537d1c8a2a; no voters:
I20260620 23:08:32.790952 4114 leader_election.cc:288] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:32.790979 4119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:32.791076 4119 raft_consensus.cc:686] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [term 1 LEADER]: Becoming Leader. State: Replica: d01730b6c042474091d386537d1c8a2a, State: Running, Role: LEADER
I20260620 23:08:32.791153 4119 consensus_queue.cc:237] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01730b6c042474091d386537d1c8a2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36315 } }
I20260620 23:08:32.791242 4114 sys_catalog.cc:564] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:32.791452 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d01730b6c042474091d386537d1c8a2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01730b6c042474091d386537d1c8a2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36315 } } }
I20260620 23:08:32.791605 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:32.791796 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d01730b6c042474091d386537d1c8a2a. Latest consensus state: current_term: 1 leader_uuid: "d01730b6c042474091d386537d1c8a2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01730b6c042474091d386537d1c8a2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36315 } } }
I20260620 23:08:32.791914 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:32.792272 4125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:32.792861 4125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:08:32 dist-test-slave-fzjm krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:32.794464 4125 catalog_manager.cc:1347] Generated new cluster ID: 8704420522da42d4a21b81f159dc8ef8
I20260620 23:08:32.795262 4125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:32.801191 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:32.787231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.1.32.126:36315)
0620 23:08:32.787427 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:32.787437 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:32.787474 (+ 37us) server_negotiation.cc:407] Connection header received
0620 23:08:32.788222 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:32.788231 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:32.788294 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:32.788443 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:32.789087 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:32.790415 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:32.791531 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:32.792093 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:32.792203 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:32.795107 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:32.795110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:32.795111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:32.795119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:32.799725 (+ 4606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:32.800130 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:32.800133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:32.800133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:32.800215 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:32.800418 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:32.800419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:32.800419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:32.800631 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:32.800794 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:32.800862 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 23:08:32.800980 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260620 23:08:32.801522 4125 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:32.802000 4125 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:32.802142 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:32.805408 4125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d01730b6c042474091d386537d1c8a2a: Generated new TSK 0
I20260620 23:08:32.805541 4125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:32.825774 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:32.821915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.1.32.126:36315)
0620 23:08:32.821966 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:32.821968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:32.822099 (+ 131us) server_negotiation.cc:407] Connection header received
0620 23:08:32.822172 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:32.822173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:32.822182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:32.822222 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:32.822650 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:32.823236 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:32.824248 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:32.824467 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:32.824515 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:32.825282 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:32.825284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:32.825284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:32.825470 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:32.825485 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:32.825490 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:32.825543 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:32.825637 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 23:08:32.825658 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 23:08:32.828094 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4046
2026-06-20T23:08:32Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1247 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:32 dist-test-slave-fzjm krb5kdc[4147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:32 dist-test-slave-fzjm krb5kdc[4147](info): set up 2 sockets
Jun 20 23:08:32 dist-test-slave-fzjm krb5kdc[4147](info): commencing operation
krb5kdc: starting...
W20260620 23:08:33.806524 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:33 dist-test-slave-fzjm krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:33.911180 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:44831
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:44831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:33.990432 4160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:33.990650 4160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:33.990685 4160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:33.991817 4160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:33.991878 4160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:33.991902 4160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:33.991931 4160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:33.991957 4160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:33.993357 4160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:44831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:44831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:33.993618 4160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:33.993815 4160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:33.998513 4165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:33.998521 4168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:33.998512 4160 server_base.cc:1034] running on GCE node
W20260620 23:08:33.998509 4166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:33.999027 4160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:33.999308 4160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:34.000469 4160 hybrid_clock.cc:648] HybridClock initialized: now 1781996914000454 us; error 26 us; skew 500 ppm
Jun 20 23:08:33 dist-test-slave-fzjm krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:34.003064 4160 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:34.003509 4160 webserver.cc:460] Webserver started at http://127.1.32.126:43563/ using document root <none> and password file <none>
I20260620 23:08:34.003777 4160 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:34.003851 4160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:34.003991 4160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:34.005262 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "2147328238084e379ab96c30c12e4cf9"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "f411380d40248a499f123bab83a8e6d3"
server_key_iv: "648ba8802092b16a09b7bb5524d2bae4"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.005657 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "2147328238084e379ab96c30c12e4cf9"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "f411380d40248a499f123bab83a8e6d3"
server_key_iv: "648ba8802092b16a09b7bb5524d2bae4"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.007025 4160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:34.007812 4175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.008003 4160 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:34.008116 4160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "2147328238084e379ab96c30c12e4cf9"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "f411380d40248a499f123bab83a8e6d3"
server_key_iv: "648ba8802092b16a09b7bb5524d2bae4"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.008190 4160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:34.035177 4160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:34.037065 4160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:34.037230 4160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:34.041339 4160 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:44831
I20260620 23:08:34.041369 4227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:44831 every 8 connection(s)
I20260620 23:08:34.041730 4160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:34.042644 4228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:34.044955 4228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9: Bootstrap starting.
I20260620 23:08:34.045193 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4160
I20260620 23:08:34.045284 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:34.045470 1153 external_mini_cluster.cc:1428] Setting key de3b12276a0ea063b5381181a982ccf9
I20260620 23:08:34.045724 4228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:34.046054 4228 log.cc:824] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:34.046756 4228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9: No bootstrap required, opened a new log
I20260620 23:08:34.048996 4228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2147328238084e379ab96c30c12e4cf9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44831 } }
I20260620 23:08:34.049170 4228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:34.049203 4228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2147328238084e379ab96c30c12e4cf9, State: Initialized, Role: FOLLOWER
I20260620 23:08:34.049314 4228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2147328238084e379ab96c30c12e4cf9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44831 } }
I20260620 23:08:34.049391 4228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:34.049417 4228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:34.049451 4228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:34.050030 4228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2147328238084e379ab96c30c12e4cf9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44831 } }
I20260620 23:08:34.050123 4228 leader_election.cc:302] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2147328238084e379ab96c30c12e4cf9; no voters:
I20260620 23:08:34.050349 4228 leader_election.cc:288] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:34.050369 4233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:34.050482 4233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [term 1 LEADER]: Becoming Leader. State: Replica: 2147328238084e379ab96c30c12e4cf9, State: Running, Role: LEADER
I20260620 23:08:34.050570 4233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2147328238084e379ab96c30c12e4cf9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44831 } }
I20260620 23:08:34.050599 4228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:34.050875 4234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2147328238084e379ab96c30c12e4cf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2147328238084e379ab96c30c12e4cf9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44831 } } }
I20260620 23:08:34.051100 4234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:34.050895 4235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2147328238084e379ab96c30c12e4cf9. Latest consensus state: current_term: 1 leader_uuid: "2147328238084e379ab96c30c12e4cf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2147328238084e379ab96c30c12e4cf9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44831 } } }
I20260620 23:08:34.051254 4235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:34.051425 4237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:34.052183 4237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:34.053741 4237 catalog_manager.cc:1347] Generated new cluster ID: 0839df6c1f594314bf3a022cf1a045d2
I20260620 23:08:34.053791 4237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:34.056669 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.046081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.1.32.126:44831)
0620 23:08:34.046249 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:34.046260 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:34.046299 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:08:34.046877 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:34.046886 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:34.046968 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:34.047095 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:34.047846 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.048659 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.049718 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.050146 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.050261 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.053338 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:34.053341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:34.053342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:34.053352 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:34.055207 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.055605 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.055608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.055608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.055693 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.055888 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.055889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.055889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.056117 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:34.056274 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:34.056352 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:08:34.056478 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260620 23:08:34.065754 4237 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:34.066382 4237 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:34.071523 4237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2147328238084e379ab96c30c12e4cf9: Generated new TSK 0
I20260620 23:08:34.071681 4237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:34.106269 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44831
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:34.198276 4254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:34.198449 4254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:34.198470 4254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:34.199560 4254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:34.199607 4254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:34.199633 4254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:34.201005 4254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:34.201174 4254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:34.201327 4254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:34.205896 4262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:34.205873 4259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:34.205915 4260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:34.206086 4254 server_base.cc:1034] running on GCE node
I20260620 23:08:34.206293 4254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:34.206521 4254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:34.207677 4254 hybrid_clock.cc:648] HybridClock initialized: now 1781996914207669 us; error 31 us; skew 500 ppm
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:34.210217 4254 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:34.210726 4254 webserver.cc:460] Webserver started at http://127.1.32.65:43667/ using document root <none> and password file <none>
I20260620 23:08:34.211014 4254 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:34.211091 4254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:34.211223 4254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:34.212275 4254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "b5b0484da8274f0aa74dc0901d9cf0d8"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "e9446d99c5684aefb4c022f6da467f90"
server_key_iv: "4cbd491fa3774c3cea60d1bc7f9c6cc1"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.212615 4254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "b5b0484da8274f0aa74dc0901d9cf0d8"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "e9446d99c5684aefb4c022f6da467f90"
server_key_iv: "4cbd491fa3774c3cea60d1bc7f9c6cc1"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.213837 4254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:34.214515 4269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.214701 4254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:34.214797 4254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "b5b0484da8274f0aa74dc0901d9cf0d8"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "e9446d99c5684aefb4c022f6da467f90"
server_key_iv: "4cbd491fa3774c3cea60d1bc7f9c6cc1"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.214879 4254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:34.238020 4254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:34.239902 4254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:34.240080 4254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:34.240343 4254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:34.240669 4254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:34.240722 4254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.240761 4254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:34.240787 4254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.246507 4254 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:37097
I20260620 23:08:34.246557 4382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:37097 every 8 connection(s)
I20260620 23:08:34.246833 4254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:34.250495 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4254
I20260620 23:08:34.250588 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:34.250782 1153 external_mini_cluster.cc:1428] Setting key c36e47b3ef4260c59eea08dcf06c55ba
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:34.255484 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.247472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:32997 (local address 127.1.32.126:44831)
0620 23:08:34.247529 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:34.247531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:34.247727 (+ 196us) server_negotiation.cc:407] Connection header received
0620 23:08:34.248211 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:34.248213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:34.248223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:34.248286 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:34.249124 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.249597 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.250551 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.250851 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.250946 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.252990 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:34.252991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:34.252992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:34.252998 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:34.254227 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.254586 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.254587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.254587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.254638 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.254871 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.254872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.254872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.254996 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:34.255080 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:34.255321 (+ 241us) server_negotiation.cc:299] Negotiation successful
0620 23:08:34.255398 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:08:34.255697 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.247468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:44831 (local address 127.1.32.65:32997)
0620 23:08:34.247629 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:34.247661 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:34.248128 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:34.248354 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:34.248357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:34.248886 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:34.249050 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:34.249055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.249668 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.249668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:34.250472 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:34.250474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.250954 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.250956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:34.251090 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.251622 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:34.251631 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:34.252899 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:34.254304 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:34.254310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:34.254312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:34.254514 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:34.254708 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:34.254709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:34.254709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:34.254809 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:34.255074 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:34.255080 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:34.255204 (+ 124us) client_negotiation.cc:769] Sending connection context
0620 23:08:34.255304 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 23:08:34.255458 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260620 23:08:34.256158 4383 heartbeater.cc:347] Connected to a master server at 127.1.32.126:44831
I20260620 23:08:34.256250 4383 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:34.256424 4383 heartbeater.cc:511] Master 127.1.32.126:44831 requested a full tablet report, sending...
I20260620 23:08:34.256863 4192 ts_manager.cc:194] Registered new tserver with Master: b5b0484da8274f0aa74dc0901d9cf0d8 (127.1.32.65:37097)
I20260620 23:08:34.257411 4192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:32997
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:34.275107 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44831
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:34.349459 4388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:34.349623 4388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:34.349644 4388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:34.350607 4388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:34.350636 4388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:34.350677 4388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:34.352104 4388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:34.352339 4388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:34.352514 4388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:34.356976 4393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:34.357141 4396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:34.357242 4388 server_base.cc:1034] running on GCE node
W20260620 23:08:34.357251 4394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:34.357494 4388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:34.357743 4388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:34.358939 4388 hybrid_clock.cc:648] HybridClock initialized: now 1781996914358924 us; error 28 us; skew 500 ppm
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:34.361294 4388 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:34.361743 4388 webserver.cc:460] Webserver started at http://127.1.32.66:37761/ using document root <none> and password file <none>
I20260620 23:08:34.361948 4388 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:34.362010 4388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:34.362151 4388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:34.363160 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "30002c0543684440be462e6e3e32214d"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "812e618cc8f1d97bc56bff9d3fe5d6e9"
server_key_iv: "1abc76100db5b1c6fa60454d0d9e26a9"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.363495 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "30002c0543684440be462e6e3e32214d"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "812e618cc8f1d97bc56bff9d3fe5d6e9"
server_key_iv: "1abc76100db5b1c6fa60454d0d9e26a9"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.364794 4388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:34.365546 4403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.365737 4388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:34.365831 4388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "30002c0543684440be462e6e3e32214d"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "812e618cc8f1d97bc56bff9d3fe5d6e9"
server_key_iv: "1abc76100db5b1c6fa60454d0d9e26a9"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.365903 4388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:34.391816 4388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:34.393875 4388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:34.394021 4388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:34.394234 4388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:34.394543 4388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:34.394600 4388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.394639 4388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:34.394666 4388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.400316 4388 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:38281
I20260620 23:08:34.400353 4516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:38281 every 8 connection(s)
I20260620 23:08:34.400624 4388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:34.408725 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.401294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:51263 (local address 127.1.32.126:44831)
0620 23:08:34.401364 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:34.401366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:34.401552 (+ 186us) server_negotiation.cc:407] Connection header received
0620 23:08:34.401979 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:34.401979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:34.401988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:34.402036 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:34.402817 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.403310 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.404289 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.404466 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.404515 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.406297 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:34.406298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:34.406299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:34.406305 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:34.407686 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.408013 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.408014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.408014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.408059 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.408248 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.408249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.408249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.408339 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:34.408385 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:34.408608 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 23:08:34.408665 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:08:34.408871 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.401272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:44831 (local address 127.1.32.66:51263)
0620 23:08:34.401438 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:34.401475 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:34.401906 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:34.402097 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:34.402100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:34.402585 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:34.402745 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:34.402749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.403380 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.403381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:34.404209 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:34.404211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.404530 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.404531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:34.404619 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.404990 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:34.404997 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:34.406218 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:34.407755 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:34.407761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:34.407762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:34.407952 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:34.408114 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:34.408114 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:34.408114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:34.408198 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:34.408390 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:34.408394 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:34.408492 (+ 98us) client_negotiation.cc:769] Sending connection context
0620 23:08:34.408568 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:08:34.408674 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260620 23:08:34.409330 4517 heartbeater.cc:347] Connected to a master server at 127.1.32.126:44831
I20260620 23:08:34.409425 4517 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:34.409633 4517 heartbeater.cc:511] Master 127.1.32.126:44831 requested a full tablet report, sending...
I20260620 23:08:34.409633 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4388
I20260620 23:08:34.409721 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:34.409893 1153 external_mini_cluster.cc:1428] Setting key ab044ba6e2dbf351ef41d5b715cffcc3
I20260620 23:08:34.410068 4192 ts_manager.cc:194] Registered new tserver with Master: 30002c0543684440be462e6e3e32214d (127.1.32.66:38281)
I20260620 23:08:34.410604 4192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:51263
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:34.431453 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44831
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:34.505900 4522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:34.506090 4522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:34.506112 4522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:34.507594 4522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:34.507642 4522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:34.507679 4522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:34.508986 4522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:34.509217 4522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:34.509428 4522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:34.513963 4528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:34.513953 4530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:34.513952 4527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:34.514003 4522 server_base.cc:1034] running on GCE node
I20260620 23:08:34.514389 4522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:34.514639 4522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:34.515801 4522 hybrid_clock.cc:648] HybridClock initialized: now 1781996914515778 us; error 34 us; skew 500 ppm
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:34.518370 4522 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:34.518776 4522 webserver.cc:460] Webserver started at http://127.1.32.67:44215/ using document root <none> and password file <none>
I20260620 23:08:34.519052 4522 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:34.519099 4522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:34.519181 4522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:34.520047 4522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "d83fc5d916e14735841d51a8d6a71972"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "c470e2f30de90fd7d1db3b3eeb6fc432"
server_key_iv: "014249a97bdbf1790be0bf6f8addef37"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.520313 4522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "d83fc5d916e14735841d51a8d6a71972"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "c470e2f30de90fd7d1db3b3eeb6fc432"
server_key_iv: "014249a97bdbf1790be0bf6f8addef37"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.521418 4522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:34.522075 4537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.522220 4522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:34.522284 4522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "d83fc5d916e14735841d51a8d6a71972"
format_stamp: "Formatted at 2026-06-20 23:08:34 on dist-test-slave-fzjm"
server_key: "c470e2f30de90fd7d1db3b3eeb6fc432"
server_key_iv: "014249a97bdbf1790be0bf6f8addef37"
server_key_version: "encryptionkey@0"
I20260620 23:08:34.522329 4522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:34.531303 4522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:34.533468 4522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:34.533572 4522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:34.533764 4522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:34.534070 4522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:34.534106 4522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.534128 4522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:34.534175 4522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:34.539778 4522 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:35437
I20260620 23:08:34.539834 4650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:35437 every 8 connection(s)
I20260620 23:08:34.540107 4522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:08:34.545316 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4522
I20260620 23:08:34.545413 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:34.545609 1153 external_mini_cluster.cc:1428] Setting key ee5ac8d927c325fdfbf11114c145ee18
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:34.548530 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.540785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:58969 (local address 127.1.32.126:44831)
0620 23:08:34.540846 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:34.540848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:34.541023 (+ 175us) server_negotiation.cc:407] Connection header received
0620 23:08:34.541468 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:34.541469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:34.541477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:34.541527 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:34.542373 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.542849 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.543867 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.544059 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.544115 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.546353 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:34.546354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:34.546354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:34.546362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:34.547537 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.547856 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.547857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.547857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.547900 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.548080 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.548081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.548081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.548168 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:34.548211 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:34.548406 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 23:08:34.548456 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:08:34.548739 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.540742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:44831 (local address 127.1.32.67:58969)
0620 23:08:34.540907 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:34.540945 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:34.541393 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:34.541584 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:34.541587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:34.542121 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:34.542307 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:34.542311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.542991 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.542992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:34.543796 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:34.543798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.544128 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.544129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:34.544220 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.544653 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:34.544661 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:34.546261 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:34.547598 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:34.547603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:34.547604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:34.547790 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:34.547951 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:34.547951 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:34.547952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:34.548034 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:34.548215 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:34.548219 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:34.548313 (+ 94us) client_negotiation.cc:769] Sending connection context
0620 23:08:34.548384 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 23:08:34.548487 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260620 23:08:34.549247 4651 heartbeater.cc:347] Connected to a master server at 127.1.32.126:44831
I20260620 23:08:34.549348 4651 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:34.549517 4651 heartbeater.cc:511] Master 127.1.32.126:44831 requested a full tablet report, sending...
I20260620 23:08:34.549880 4192 ts_manager.cc:194] Registered new tserver with Master: d83fc5d916e14735841d51a8d6a71972 (127.1.32.67:35437)
I20260620 23:08:34.550206 4192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:58969
I20260620 23:08:34.557235 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996914, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:34.580531 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:34.574322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55814 (local address 127.1.32.126:44831)
0620 23:08:34.574365 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:34.574367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:34.574483 (+ 116us) server_negotiation.cc:407] Connection header received
0620 23:08:34.574530 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:34.574531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:34.574538 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:34.574589 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:34.575174 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.575638 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.576310 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:34.576474 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:34.576520 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:34.578471 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:34.578472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:34.578472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:34.578478 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:34.579680 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.579957 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.579958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.579958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.580006 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:34.580144 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:34.580145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:34.580145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:34.580257 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:34.580322 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:34.580400 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:08:34.580449 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 23:08:34.580775 4192 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55814
I20260620 23:08:34.581224 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4254
I20260620 23:08:34.585726 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4388
I20260620 23:08:34.589618 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4522
I20260620 23:08:34.593447 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4160
2026-06-20T23:08:34Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1766 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4663](info): set up 2 sockets
Jun 20 23:08:34 dist-test-slave-fzjm krb5kdc[4663](info): commencing operation
krb5kdc: starting...
W20260620 23:08:35.570403 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:35 dist-test-slave-fzjm krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:35.675660 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46859
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:35.753839 4676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:35.754027 4676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:35.754048 4676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:35.755213 4676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:35.755264 4676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:35.755281 4676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:35.755329 4676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:35.755376 4676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:35.756884 4676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:35.757117 4676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:35.757342 4676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:35.762095 4681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:35.762130 4676 server_base.cc:1034] running on GCE node
W20260620 23:08:35.762100 4682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:35.762109 4684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:35.762593 4676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:35.762938 4676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:35.764104 4676 hybrid_clock.cc:648] HybridClock initialized: now 1781996915764092 us; error 48 us; skew 500 ppm
Jun 20 23:08:35 dist-test-slave-fzjm krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996915, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:35.770802 4676 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:35.771296 4676 webserver.cc:460] Webserver started at http://127.1.32.126:38763/ using document root <none> and password file <none>
I20260620 23:08:35.771513 4676 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:35.771579 4676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:35.771713 4676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:35.772660 4676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "4629e838299b497598c195d46e813d2a"
format_stamp: "Formatted at 2026-06-20 23:08:35 on dist-test-slave-fzjm"
server_key: "586bfcccd816b24d60224c062c526d79"
server_key_iv: "d29dd4a31bc5e65e0e781e950718ee33"
server_key_version: "encryptionkey@0"
I20260620 23:08:35.772987 4676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "4629e838299b497598c195d46e813d2a"
format_stamp: "Formatted at 2026-06-20 23:08:35 on dist-test-slave-fzjm"
server_key: "586bfcccd816b24d60224c062c526d79"
server_key_iv: "d29dd4a31bc5e65e0e781e950718ee33"
server_key_version: "encryptionkey@0"
I20260620 23:08:35.774219 4676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:08:35.775025 4691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:35.775215 4676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:08:35.775303 4676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "4629e838299b497598c195d46e813d2a"
format_stamp: "Formatted at 2026-06-20 23:08:35 on dist-test-slave-fzjm"
server_key: "586bfcccd816b24d60224c062c526d79"
server_key_iv: "d29dd4a31bc5e65e0e781e950718ee33"
server_key_version: "encryptionkey@0"
I20260620 23:08:35.775385 4676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:35.788358 4676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:35.790125 4676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:35.790262 4676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:35.793939 4676 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:46859
I20260620 23:08:35.793967 4743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:46859 every 8 connection(s)
I20260620 23:08:35.794387 4676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:35.795199 4744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:35.797425 4744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a: Bootstrap starting.
I20260620 23:08:35.798012 4744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:35.798295 4744 log.cc:824] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:35.799021 4744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a: No bootstrap required, opened a new log
I20260620 23:08:35.800097 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4676
I20260620 23:08:35.800211 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:35.800379 1153 external_mini_cluster.cc:1428] Setting key 7241d6e6f23c98674a08662c06784753
I20260620 23:08:35.800803 4744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4629e838299b497598c195d46e813d2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46859 } }
I20260620 23:08:35.800964 4744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:35.801035 4744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4629e838299b497598c195d46e813d2a, State: Initialized, Role: FOLLOWER
I20260620 23:08:35.801168 4744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4629e838299b497598c195d46e813d2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46859 } }
I20260620 23:08:35.801252 4744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:35.801276 4744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:35.801348 4744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:35.801867 4744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4629e838299b497598c195d46e813d2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46859 } }
I20260620 23:08:35.801980 4744 leader_election.cc:302] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4629e838299b497598c195d46e813d2a; no voters:
I20260620 23:08:35.802145 4744 leader_election.cc:288] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:35.802170 4749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:35.802330 4749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [term 1 LEADER]: Becoming Leader. State: Replica: 4629e838299b497598c195d46e813d2a, State: Running, Role: LEADER
I20260620 23:08:35.802443 4744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:35.802413 4749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4629e838299b497598c195d46e813d2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46859 } }
I20260620 23:08:35.802825 4751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4629e838299b497598c195d46e813d2a. Latest consensus state: current_term: 1 leader_uuid: "4629e838299b497598c195d46e813d2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4629e838299b497598c195d46e813d2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46859 } } }
I20260620 23:08:35.802953 4751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:35.803159 4755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:35.803133 4750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4629e838299b497598c195d46e813d2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4629e838299b497598c195d46e813d2a" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46859 } } }
I20260620 23:08:35.803229 4750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:35.803769 4755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:35.805769 4755 catalog_manager.cc:1347] Generated new cluster ID: c4b9fec40cde44f1ad3f5763b923cfb2
I20260620 23:08:35.805817 4755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:35 dist-test-slave-fzjm krb5kdc[4663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:35.813300 4748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:35.800880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48084 (local address 127.1.32.126:46859)
0620 23:08:35.804077 (+ 3197us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:35.804086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:35.804122 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:08:35.804443 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:35.804452 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:35.804523 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:35.804671 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:35.805432 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:35.806438 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:35.807310 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:35.807733 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:35.807848 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:35.810107 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:35.810111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:35.810111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:35.810129 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:35.811739 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:35.812171 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:35.812174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:35.812174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:35.812259 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:35.812478 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:35.812479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:35.812479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:35.812695 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:35.812868 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:35.812941 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:08:35.813095 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3080,"thread_start_us":515,"threads_started":1}
I20260620 23:08:35.818964 4755 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:35.819422 4755 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:35.822307 4755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4629e838299b497598c195d46e813d2a: Generated new TSK 0
I20260620 23:08:35.822464 4755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:35.853843 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46859
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:35.941226 4770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:35.941416 4770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:35.941438 4770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:35.942559 4770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:35.942603 4770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:35.942631 4770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:35.944140 4770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:35.944411 4770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:35.944674 4770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:35.949549 4775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:35.949674 4770 server_base.cc:1034] running on GCE node
W20260620 23:08:35.949568 4778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:35.949535 4776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:35.949952 4770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:35.950212 4770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:35.951370 4770 hybrid_clock.cc:648] HybridClock initialized: now 1781996915951358 us; error 44 us; skew 500 ppm
Jun 20 23:08:35 dist-test-slave-fzjm krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996915, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:35.953665 4770 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:35.954073 4770 webserver.cc:460] Webserver started at http://127.1.32.65:33043/ using document root <none> and password file <none>
I20260620 23:08:35.954298 4770 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:35.954358 4770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:35.954495 4770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:35.955483 4770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "ba549f14bc9b49149a8625658fe3f067"
format_stamp: "Formatted at 2026-06-20 23:08:35 on dist-test-slave-fzjm"
server_key: "3805b1d39800314ade0530e8204b5c94"
server_key_iv: "b49ae16f5efc4ff7052f3af716a86559"
server_key_version: "encryptionkey@0"
I20260620 23:08:35.955819 4770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "ba549f14bc9b49149a8625658fe3f067"
format_stamp: "Formatted at 2026-06-20 23:08:35 on dist-test-slave-fzjm"
server_key: "3805b1d39800314ade0530e8204b5c94"
server_key_iv: "b49ae16f5efc4ff7052f3af716a86559"
server_key_version: "encryptionkey@0"
I20260620 23:08:35.957031 4770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:35.957695 4785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:35.957860 4770 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:35.957955 4770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "ba549f14bc9b49149a8625658fe3f067"
format_stamp: "Formatted at 2026-06-20 23:08:35 on dist-test-slave-fzjm"
server_key: "3805b1d39800314ade0530e8204b5c94"
server_key_iv: "b49ae16f5efc4ff7052f3af716a86559"
server_key_version: "encryptionkey@0"
I20260620 23:08:35.958035 4770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:35.976178 4770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:35.978132 4770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:35.978322 4770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:35.978574 4770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:35.978987 4770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:35.979049 4770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:35.979094 4770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:35.979126 4770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:35.985344 4770 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:36583
I20260620 23:08:35.985401 4898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:36583 every 8 connection(s)
I20260620 23:08:35.985770 4770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:35.988862 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4770
I20260620 23:08:35.989002 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:35.989209 1153 external_mini_cluster.cc:1428] Setting key 122f9bf9b22a1b60f42f1ac20a6176be
Jun 20 23:08:35 dist-test-slave-fzjm krb5kdc[4663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996915, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:35.995828 4748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:35.986544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:44921 (local address 127.1.32.126:46859)
0620 23:08:35.986608 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:35.986610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:35.986812 (+ 202us) server_negotiation.cc:407] Connection header received
0620 23:08:35.987830 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:35.987832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:35.987848 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:35.987898 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:35.988486 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:35.989129 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:35.990373 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:35.990597 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:35.990682 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:35.993006 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:35.993008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:35.993009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:35.993030 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:35.994407 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:35.994863 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:35.994864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:35.994865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:35.994937 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:35.995196 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:35.995197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:35.995198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:35.995310 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:35.995384 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:35.995652 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 23:08:35.995740 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:08:35.995991 4901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:35.986516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46859 (local address 127.1.32.65:44921)
0620 23:08:35.986690 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:35.986733 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:35.987711 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:35.987969 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:35.987973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:35.988217 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:35.988391 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:35.988395 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:35.989232 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:35.989233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:35.990277 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:35.990280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:35.990687 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:35.990688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:35.990822 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:35.991396 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:35.991407 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:35.992913 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:35.994549 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:35.994555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:35.994557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:35.994789 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:35.995014 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:35.995015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:35.995015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:35.995128 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:35.995375 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:35.995382 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:35.995515 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 23:08:35.995628 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 23:08:35.995770 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260620 23:08:35.996456 4899 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46859
I20260620 23:08:35.996543 4899 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:35.996716 4899 heartbeater.cc:511] Master 127.1.32.126:46859 requested a full tablet report, sending...
I20260620 23:08:35.997304 4708 ts_manager.cc:194] Registered new tserver with Master: ba549f14bc9b49149a8625658fe3f067 (127.1.32.65:36583)
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:08:35.998211 4708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:44921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:36.013859 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46859
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:36.085449 4904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:36.085633 4904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:36.085655 4904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:36.086684 4904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:36.086719 4904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:36.086742 4904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:36.088181 4904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:36.088357 4904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:36.088495 4904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:36.093255 4909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:36.093382 4904 server_base.cc:1034] running on GCE node
W20260620 23:08:36.093297 4912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:36.093262 4910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:36.093797 4904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:36.094053 4904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:36.095216 4904 hybrid_clock.cc:648] HybridClock initialized: now 1781996916095202 us; error 30 us; skew 500 ppm
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:36.097740 4904 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:36.098155 4904 webserver.cc:460] Webserver started at http://127.1.32.66:41019/ using document root <none> and password file <none>
I20260620 23:08:36.098371 4904 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:36.098413 4904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:36.098549 4904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:36.099567 4904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "01ea2fc6514c4e019590a0a0e4d1f7e2"
format_stamp: "Formatted at 2026-06-20 23:08:36 on dist-test-slave-fzjm"
server_key: "97bc37dc7a7d5c5d726cc18c63f0a206"
server_key_iv: "3a0baa4936c18fee7d33233e17e5703b"
server_key_version: "encryptionkey@0"
I20260620 23:08:36.099931 4904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "01ea2fc6514c4e019590a0a0e4d1f7e2"
format_stamp: "Formatted at 2026-06-20 23:08:36 on dist-test-slave-fzjm"
server_key: "97bc37dc7a7d5c5d726cc18c63f0a206"
server_key_iv: "3a0baa4936c18fee7d33233e17e5703b"
server_key_version: "encryptionkey@0"
I20260620 23:08:36.101145 4904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:36.101801 4919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:36.102061 4904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:36.102150 4904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "01ea2fc6514c4e019590a0a0e4d1f7e2"
format_stamp: "Formatted at 2026-06-20 23:08:36 on dist-test-slave-fzjm"
server_key: "97bc37dc7a7d5c5d726cc18c63f0a206"
server_key_iv: "3a0baa4936c18fee7d33233e17e5703b"
server_key_version: "encryptionkey@0"
I20260620 23:08:36.102218 4904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:36.119886 4904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:36.121495 4904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:36.121644 4904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:36.121835 4904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:36.122109 4904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:36.122166 4904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:36.122202 4904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:36.122227 4904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:36.127692 4904 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:40771
I20260620 23:08:36.127754 5032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:40771 every 8 connection(s)
I20260620 23:08:36.128053 4904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[4663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:36.136648 4748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:36.128720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:54743 (local address 127.1.32.126:46859)
0620 23:08:36.128768 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:36.128770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:36.128978 (+ 208us) server_negotiation.cc:407] Connection header received
0620 23:08:36.129776 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:36.129777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:36.129787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:36.129834 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:36.130334 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.130777 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.131954 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.132253 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.132326 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:36.134168 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:36.134169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:36.134169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:36.134176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:36.135604 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.135935 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.135936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.135937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.135982 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.136178 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.136179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.136179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.136268 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:36.136314 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:36.136510 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 23:08:36.136589 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:36.136790 5035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:36.128702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46859 (local address 127.1.32.66:54743)
0620 23:08:36.128875 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:36.128906 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:36.129700 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:36.129882 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:36.129885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:36.130111 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:36.130273 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:36.130277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.130844 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.130844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:36.131825 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:36.131827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.132339 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.132340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:36.132446 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:36.132852 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:36.132858 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:36.134088 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:36.135669 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:36.135673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:36.135675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:36.135869 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:36.136040 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:36.136040 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:36.136040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:36.136129 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:36.136317 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:36.136320 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:36.136415 (+ 95us) client_negotiation.cc:769] Sending connection context
0620 23:08:36.136487 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:08:36.136598 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260620 23:08:36.137266 5033 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46859
I20260620 23:08:36.137336 5033 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:36.137495 5033 heartbeater.cc:511] Master 127.1.32.126:46859 requested a full tablet report, sending...
I20260620 23:08:36.137840 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 4904
I20260620 23:08:36.137841 4708 ts_manager.cc:194] Registered new tserver with Master: 01ea2fc6514c4e019590a0a0e4d1f7e2 (127.1.32.66:40771)
I20260620 23:08:36.137933 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:36.138127 1153 external_mini_cluster.cc:1428] Setting key bd961df6505776775846eba649da882c
I20260620 23:08:36.138273 4708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:54743
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:36.160827 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46859
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:36.235351 5038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:36.235584 5038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:36.235620 5038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:36.236701 5038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:36.236759 5038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:36.236797 5038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:36.238112 5038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:36.238384 5038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:36.238600 5038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:36.243482 5043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:36.243491 5044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:36.243551 5038 server_base.cc:1034] running on GCE node
W20260620 23:08:36.243497 5046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:36.243877 5038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:36.244150 5038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:36.245340 5038 hybrid_clock.cc:648] HybridClock initialized: now 1781996916245314 us; error 35 us; skew 500 ppm
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:36.247906 5038 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:36.248266 5038 webserver.cc:460] Webserver started at http://127.1.32.67:37141/ using document root <none> and password file <none>
I20260620 23:08:36.248426 5038 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:36.248462 5038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:36.248543 5038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:36.249620 5038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "5c70032383bd4cea949e8e47ff1bf50c"
format_stamp: "Formatted at 2026-06-20 23:08:36 on dist-test-slave-fzjm"
server_key: "f1514f620f2d0a08dbbc8327b6c7ea01"
server_key_iv: "a9244d055dd3766e71ab1d29c5840e47"
server_key_version: "encryptionkey@0"
I20260620 23:08:36.249908 5038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "5c70032383bd4cea949e8e47ff1bf50c"
format_stamp: "Formatted at 2026-06-20 23:08:36 on dist-test-slave-fzjm"
server_key: "f1514f620f2d0a08dbbc8327b6c7ea01"
server_key_iv: "a9244d055dd3766e71ab1d29c5840e47"
server_key_version: "encryptionkey@0"
I20260620 23:08:36.251119 5038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:36.251734 5053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:36.251896 5038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:36.251978 5038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "5c70032383bd4cea949e8e47ff1bf50c"
format_stamp: "Formatted at 2026-06-20 23:08:36 on dist-test-slave-fzjm"
server_key: "f1514f620f2d0a08dbbc8327b6c7ea01"
server_key_iv: "a9244d055dd3766e71ab1d29c5840e47"
server_key_version: "encryptionkey@0"
I20260620 23:08:36.252029 5038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:36.268115 5038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:36.270306 5038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:36.270411 5038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:36.270587 5038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:36.270816 5038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:36.270844 5038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:36.270865 5038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:36.270877 5038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:36.276450 5038 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:41571
I20260620 23:08:36.276547 5166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:41571 every 8 connection(s)
I20260620 23:08:36.276826 5038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[4663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996916, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:36.284902 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5038
I20260620 23:08:36.284982 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:36.285189 1153 external_mini_cluster.cc:1428] Setting key db7b654825072022f196a90d9cedc02b
I20260620 23:08:36.286420 4748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:36.277557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:34855 (local address 127.1.32.126:46859)
0620 23:08:36.277623 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:36.277625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:36.277840 (+ 215us) server_negotiation.cc:407] Connection header received
0620 23:08:36.278858 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:36.278859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:36.278872 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:36.278962 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:36.279585 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.280099 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.281158 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.281368 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.281455 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:36.283500 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:36.283501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:36.283502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:36.283508 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:36.284933 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.285409 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.285410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.285411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.285471 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.285760 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.285761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.285761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.285927 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:36.285996 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:36.286305 (+ 309us) server_negotiation.cc:299] Negotiation successful
0620 23:08:36.286364 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 23:08:36.286556 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:36.277548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46859 (local address 127.1.32.67:34855)
0620 23:08:36.277727 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:36.277761 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:36.278746 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:36.279036 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:36.279040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:36.279281 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:36.279470 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:36.279474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.280198 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.280200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:36.281041 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:36.281043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.281461 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.281462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:36.281595 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:36.282006 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:36.282014 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:36.283399 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:36.285034 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:36.285040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:36.285042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:36.285336 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:36.285547 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:36.285548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:36.285548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:36.285685 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:36.286000 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:36.286005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:36.286171 (+ 166us) client_negotiation.cc:769] Sending connection context
0620 23:08:36.286275 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:08:36.286380 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260620 23:08:36.287102 5167 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46859
I20260620 23:08:36.287213 5167 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:36.287374 5167 heartbeater.cc:511] Master 127.1.32.126:46859 requested a full tablet report, sending...
I20260620 23:08:36.287735 4708 ts_manager.cc:194] Registered new tserver with Master: 5c70032383bd4cea949e8e47ff1bf50c (127.1.32.67:41571)
I20260620 23:08:36.288096 4708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:34855
I20260620 23:08:36.296833 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[4663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[4663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:36.319214 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:36.312026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.1.32.65:36583)
0620 23:08:36.312184 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:36.312189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:36.312199 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:36.312224 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:36.312226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:36.312239 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:36.312304 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:36.312840 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.313575 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.314235 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.314559 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.314611 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:36.316684 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:36.316690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:36.316690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:36.316696 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:36.318137 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.318445 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.318447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.318448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.318511 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.318663 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.318663 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.318663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.318786 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:36.318939 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:36.319014 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:08:36.319078 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[4663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:36.326545 4748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:36.320412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.1.32.126:46859)
0620 23:08:36.320464 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:36.320466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:36.320502 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:08:36.320575 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:36.320575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:36.320583 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:36.320628 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:36.321035 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.321477 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.322115 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:36.322289 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:36.322350 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:36.324446 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:36.324447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:36.324447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:36.324454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:36.325637 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.325935 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.325936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.325936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.325990 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:36.326150 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:36.326151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:36.326151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:36.326275 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:36.326344 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:36.326422 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:08:36.326485 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 23:08:36.326767 4708 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48096
I20260620 23:08:36.327143 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4770
I20260620 23:08:36.331362 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4904
I20260620 23:08:36.335814 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5038
I20260620 23:08:36.340168 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 4676
2026-06-20T23:08:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1747 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[5179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[5179](info): set up 2 sockets
Jun 20 23:08:36 dist-test-slave-fzjm krb5kdc[5179](info): commencing operation
krb5kdc: starting...
W20260620 23:08:37.330654 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:37.437462 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45333
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:45333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:37.514571 5192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:37.514842 5192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:37.514865 5192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:37.515982 5192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:37.516024 5192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:37.516039 5192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:37.516052 5192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:37.516064 5192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:37.517364 5192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:45333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:37.517553 5192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:37.517710 5192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:37.522429 5197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.522456 5200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.522466 5198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:37.522598 5192 server_base.cc:1034] running on GCE node
I20260620 23:08:37.522856 5192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:37.523229 5192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:37.524405 5192 hybrid_clock.cc:648] HybridClock initialized: now 1781996917524395 us; error 46 us; skew 500 ppm
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:37.526779 5192 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:37.527215 5192 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:37.527285 5192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:37.527436 5192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:37.528455 5192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "84ee4cd41b284116ac26fa3b543aaade"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "1a8674182ec7073ea73097ab362548aa"
server_key_iv: "469a8f798b2101e48efa304579083792"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.528815 5192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "84ee4cd41b284116ac26fa3b543aaade"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "1a8674182ec7073ea73097ab362548aa"
server_key_iv: "469a8f798b2101e48efa304579083792"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.530036 5192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:37.530800 5206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:37.530997 5192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:08:37.531096 5192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "84ee4cd41b284116ac26fa3b543aaade"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "1a8674182ec7073ea73097ab362548aa"
server_key_iv: "469a8f798b2101e48efa304579083792"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.531176 5192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:37.537564 5192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:37.539331 5192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:37.539484 5192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:37.543327 5192 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:45333
I20260620 23:08:37.543352 5258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:45333 every 8 connection(s)
I20260620 23:08:37.543612 5192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:37.544451 5259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:37.546506 5259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade: Bootstrap starting.
I20260620 23:08:37.547092 5259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:37.547324 5259 log.cc:824] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:37.547915 5259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade: No bootstrap required, opened a new log
I20260620 23:08:37.549378 5259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ee4cd41b284116ac26fa3b543aaade" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45333 } }
I20260620 23:08:37.549507 5259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:37.549530 5259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84ee4cd41b284116ac26fa3b543aaade, State: Initialized, Role: FOLLOWER
I20260620 23:08:37.549786 5259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ee4cd41b284116ac26fa3b543aaade" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45333 } }
I20260620 23:08:37.549868 5259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:37.549890 5259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:37.549918 5259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:37.550657 5259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ee4cd41b284116ac26fa3b543aaade" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45333 } }
I20260620 23:08:37.550740 5259 leader_election.cc:302] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84ee4cd41b284116ac26fa3b543aaade; no voters:
I20260620 23:08:37.550848 5259 leader_election.cc:288] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:37.550945 5262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:37.551144 5262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [term 1 LEADER]: Becoming Leader. State: Replica: 84ee4cd41b284116ac26fa3b543aaade, State: Running, Role: LEADER
I20260620 23:08:37.551177 5259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:37.551234 5262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ee4cd41b284116ac26fa3b543aaade" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45333 } }
I20260620 23:08:37.551267 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5192
I20260620 23:08:37.551386 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:37.551564 1153 external_mini_cluster.cc:1428] Setting key 30ac5e3204ed2d148d1abd811c0f6280
I20260620 23:08:37.551607 5263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84ee4cd41b284116ac26fa3b543aaade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ee4cd41b284116ac26fa3b543aaade" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45333 } } }
I20260620 23:08:37.551600 5264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84ee4cd41b284116ac26fa3b543aaade. Latest consensus state: current_term: 1 leader_uuid: "84ee4cd41b284116ac26fa3b543aaade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ee4cd41b284116ac26fa3b543aaade" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45333 } } }
I20260620 23:08:37.551697 5263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:37.551702 5264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade [sys.catalog]: This master's current role is: LEADER
W20260620 23:08:37.553498 5280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:37.553571 5280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:37.553658 5270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:37.553858 5270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:37.555668 5270 catalog_manager.cc:1347] Generated new cluster ID: 3a50174eb3b74e75870dd408cdb7674d
I20260620 23:08:37.555730 5270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:37.564136 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:37.552369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.1.32.126:45333)
0620 23:08:37.552589 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:37.552599 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:37.552633 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:08:37.552790 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:37.552797 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:37.552852 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:37.553000 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:37.553841 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.555258 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.556353 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.556936 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.557356 (+ 420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:37.559912 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:37.559916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:37.559917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:37.559928 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:37.562062 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:37.562518 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:37.562521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:37.562522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:37.562643 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:37.562872 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:37.562873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:37.562873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:37.563199 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:37.563694 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:37.563760 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 23:08:37.563884 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":67,"threads_started":1}
I20260620 23:08:37.565930 5270 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:37.566589 5270 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:37.575953 5270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade: Generated new TSK 0
I20260620 23:08:37.576120 5270 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:37.601737 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45333
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:37.680110 5285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:37.680302 5285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:37.680325 5285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:37.681440 5285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:37.681480 5285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:37.681506 5285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:37.682950 5285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:37.683169 5285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:37.683403 5285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:37.688207 5290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.688206 5291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.688243 5293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:37.688393 5285 server_base.cc:1034] running on GCE node
I20260620 23:08:37.688619 5285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:37.688884 5285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:37.690021 5285 hybrid_clock.cc:648] HybridClock initialized: now 1781996917690009 us; error 28 us; skew 500 ppm
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:37.692469 5285 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:37.692824 5285 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:37.692895 5285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:37.693046 5285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:37.694067 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "6cb1da398494425880f05351d6c5d01e"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "bdb5e746f99eb6362ab506558cdebb70"
server_key_iv: "aebb1ce5495c0d4ccf758edb7aea1045"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.694433 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "6cb1da398494425880f05351d6c5d01e"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "bdb5e746f99eb6362ab506558cdebb70"
server_key_iv: "aebb1ce5495c0d4ccf758edb7aea1045"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.695823 5285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:37.696524 5299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:37.696676 5285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:37.696779 5285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "6cb1da398494425880f05351d6c5d01e"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "bdb5e746f99eb6362ab506558cdebb70"
server_key_iv: "aebb1ce5495c0d4ccf758edb7aea1045"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.696849 5285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:37.709831 5285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:37.711812 5285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:37.711967 5285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:37.712195 5285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:37.712486 5285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:37.712543 5285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:37.712582 5285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:37.712610 5285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:37.718286 5285 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:40231
I20260620 23:08:37.718328 5412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:40231 every 8 connection(s)
I20260620 23:08:37.718569 5285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:37.726065 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5285
I20260620 23:08:37.726152 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:37.726361 1153 external_mini_cluster.cc:1428] Setting key 979fcd6cd3b49c1c009f2c7fa6f4915a
I20260620 23:08:37.727603 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:37.719266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:42375 (local address 127.1.32.126:45333)
0620 23:08:37.719315 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:37.719317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:37.719545 (+ 228us) server_negotiation.cc:407] Connection header received
0620 23:08:37.719976 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:37.719977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:37.719990 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:37.720054 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:37.720902 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.721391 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.722349 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.722542 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.722595 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:37.724729 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:37.724730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:37.724730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:37.724736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:37.726189 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:37.726621 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:37.726622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:37.726623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:37.726690 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:37.726944 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:37.726945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:37.726945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:37.727075 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:37.727153 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:37.727408 (+ 255us) server_negotiation.cc:299] Negotiation successful
0620 23:08:37.727488 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:08:37.727751 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:37.719261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45333 (local address 127.1.32.65:42375)
0620 23:08:37.719425 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:37.719459 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:37.719901 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:37.720107 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:37.720109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:37.720345 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:37.720831 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:37.720836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.721467 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.721468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:37.722274 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:37.722276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.722612 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.722613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:37.722707 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:37.723251 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:37.723259 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:37.724641 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:37.726295 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:37.726300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:37.726302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:37.726550 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:37.726736 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:37.726736 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:37.726737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:37.726857 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:37.727145 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:37.727151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:37.727280 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 23:08:37.727386 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 23:08:37.727533 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260620 23:08:37.728456 5413 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45333
I20260620 23:08:37.728569 5413 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:37.728709 5413 heartbeater.cc:511] Master 127.1.32.126:45333 requested a full tablet report, sending...
I20260620 23:08:37.729224 5223 ts_manager.cc:194] Registered new tserver with Master: 6cb1da398494425880f05351d6c5d01e (127.1.32.65:40231)
I20260620 23:08:37.729809 5223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:42375
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:37.749346 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45333
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:37.827049 5418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:37.827219 5418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:37.827239 5418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:37.828222 5418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:37.828253 5418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:37.828275 5418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:37.829527 5418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:37.829684 5418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:37.829829 5418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:37.834398 5424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.834404 5423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.834461 5426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:37.834688 5418 server_base.cc:1034] running on GCE node
I20260620 23:08:37.834867 5418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:37.835134 5418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:37.836293 5418 hybrid_clock.cc:648] HybridClock initialized: now 1781996917836279 us; error 25 us; skew 500 ppm
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:37.838690 5418 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:37.839099 5418 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:37.839170 5418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:37.839318 5418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:37.840310 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "68a29fb187714276936c7e4e50210192"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "c75d2ae3a63ccccdfeb304fe11efc3a1"
server_key_iv: "a0c51eeea8156788c435c3bc1170f8da"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.840677 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "68a29fb187714276936c7e4e50210192"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "c75d2ae3a63ccccdfeb304fe11efc3a1"
server_key_iv: "a0c51eeea8156788c435c3bc1170f8da"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.841919 5418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:37.842646 5432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:37.842814 5418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:37.842929 5418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "68a29fb187714276936c7e4e50210192"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "c75d2ae3a63ccccdfeb304fe11efc3a1"
server_key_iv: "a0c51eeea8156788c435c3bc1170f8da"
server_key_version: "encryptionkey@0"
I20260620 23:08:37.843027 5418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:37.872898 5418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:37.875445 5418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:37.875620 5418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:37.875896 5418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:37.876216 5418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:37.876271 5418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:37.876312 5418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:37.876353 5418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:37.882344 5418 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:41673
I20260620 23:08:37.882383 5545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:41673 every 8 connection(s)
I20260620 23:08:37.882674 5418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:37.884058 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5418
I20260620 23:08:37.884186 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:37.884394 1153 external_mini_cluster.cc:1428] Setting key ed7700c98c16e6e7d4992ed43bc5e98b
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:37.893121 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:37.883398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:49869 (local address 127.1.32.126:45333)
0620 23:08:37.883462 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:37.883464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:37.883690 (+ 226us) server_negotiation.cc:407] Connection header received
0620 23:08:37.884170 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:37.884171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:37.884184 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:37.884252 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:37.885511 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.886064 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.887650 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.887968 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.888039 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:37.890167 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:37.890169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:37.890169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:37.890178 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:37.891809 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:37.892205 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:37.892206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:37.892207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:37.892268 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:37.892492 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:37.892493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:37.892493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:37.892633 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:37.892705 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:37.892926 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 23:08:37.893020 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:08:37.893185 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:37.883396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45333 (local address 127.1.32.66:49869)
0620 23:08:37.883560 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:37.883600 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:37.884068 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:37.884328 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:37.884332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:37.884750 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:37.885413 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:37.885418 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.886167 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.886168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:37.887557 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:37.887559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:37.888055 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:37.888056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:37.888171 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:37.888679 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:37.888687 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:37.890069 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:37.891898 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:37.891903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:37.891905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:37.892123 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:37.892337 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:37.892338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:37.892338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:37.892427 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:37.892696 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:37.892700 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:37.892802 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 23:08:37.892879 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 23:08:37.892989 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260620 23:08:37.893662 5546 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45333
I20260620 23:08:37.893766 5546 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:37.893908 5546 heartbeater.cc:511] Master 127.1.32.126:45333 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
I20260620 23:08:37.894313 5223 ts_manager.cc:194] Registered new tserver with Master: 68a29fb187714276936c7e4e50210192 (127.1.32.66:41673)
I20260620 23:08:37.894698 5223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:49869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:37.909633 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45333
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:37.990471 5551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:37.990674 5551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:37.990696 5551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:37.991859 5551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:37.991896 5551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:37.991919 5551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:37.993244 5551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:37.993414 5551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:37.993574 5551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:37.998018 5557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.998010 5556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:37.998010 5559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:37.998147 5551 server_base.cc:1034] running on GCE node
I20260620 23:08:37.998389 5551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:37.998633 5551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:37.999789 5551 hybrid_clock.cc:648] HybridClock initialized: now 1781996917999769 us; error 34 us; skew 500 ppm
Jun 20 23:08:37 dist-test-slave-fzjm krb5kdc[5179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:38.002102 5551 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:38.002408 5551 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:38.002470 5551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:38.002589 5551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:38.003535 5551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "dafa07fad5b447109b3063dd1f908c9e"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "14d1fd7563057bd7591ab735e1d725bb"
server_key_iv: "683f635dbaba9149615484fd77c8963d"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.003916 5551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "dafa07fad5b447109b3063dd1f908c9e"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "14d1fd7563057bd7591ab735e1d725bb"
server_key_iv: "683f635dbaba9149615484fd77c8963d"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.005189 5551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:08:38.005885 5565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.006067 5551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:38.006156 5551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "dafa07fad5b447109b3063dd1f908c9e"
format_stamp: "Formatted at 2026-06-20 23:08:37 on dist-test-slave-fzjm"
server_key: "14d1fd7563057bd7591ab735e1d725bb"
server_key_iv: "683f635dbaba9149615484fd77c8963d"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.006239 5551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:38.037595 5551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:38.039487 5551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:38.039647 5551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:38.039881 5551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:38.040210 5551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:38.040282 5551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.040319 5551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:38.040367 5551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.046289 5551 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:38523
I20260620 23:08:38.046319 5678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:38523 every 8 connection(s)
I20260620 23:08:38.046636 5551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:38 dist-test-slave-fzjm krb5kdc[5179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996917, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:38.054124 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5551
I20260620 23:08:38.054240 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:38.054418 1153 external_mini_cluster.cc:1428] Setting key 3efbd75f492f51fd73309d1fcbfd0f91
I20260620 23:08:38.056051 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.047377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:39977 (local address 127.1.32.126:45333)
0620 23:08:38.047432 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.047433 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.047637 (+ 204us) server_negotiation.cc:407] Connection header received
0620 23:08:38.048095 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.048097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.048108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.048151 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.049032 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.049510 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.050530 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.050708 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.050783 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.053130 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.053131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.053131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:38.053139 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.054785 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:38.055260 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:38.055261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:38.055262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:38.055328 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:38.055595 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:38.055596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:38.055596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:38.055694 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.055747 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.055926 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.055980 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:08:38.056210 5681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.047353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45333 (local address 127.1.32.67:39977)
0620 23:08:38.047533 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.047562 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.048001 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.048229 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.048233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.048481 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.048936 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.048940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.049608 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.049609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.050428 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.050429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.050792 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.050793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.050992 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.051496 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:38.051510 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.053005 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.054856 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:38.054862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:38.054864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:38.055170 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:38.055395 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:38.055396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:38.055396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:38.055536 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:38.055741 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.055743 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.055843 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.055915 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.056027 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260620 23:08:38.056633 5679 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45333
I20260620 23:08:38.056739 5679 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:38.056877 5679 heartbeater.cc:511] Master 127.1.32.126:45333 requested a full tablet report, sending...
I20260620 23:08:38.057240 5223 ts_manager.cc:194] Registered new tserver with Master: dafa07fad5b447109b3063dd1f908c9e (127.1.32.67:38523)
I20260620 23:08:38.057578 5223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:39977
I20260620 23:08:38.066264 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:08:38.072726 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.067524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.1.32.126:45333)
0620 23:08:38.067582 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.067584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.067637 (+ 53us) server_negotiation.cc:407] Connection header received
0620 23:08:38.067708 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.067709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.067718 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.067771 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.068277 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.068708 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.069365 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.069552 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.069630 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.070602 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.070604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.070604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:38.070611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.071641 (+ 1030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:38.071960 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:38.071961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:38.071961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:38.072030 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:38.072213 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:38.072214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:38.072214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:38.072349 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.072463 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.072566 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.072637 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:08:38.073915 5223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:38.075536 5223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:38.081213 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.077680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.1.32.66:41673)
0620 23:08:38.077829 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.077834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.077845 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:38.077888 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.077890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.077906 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.077984 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:38.078325 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.079080 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.080324 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.080817 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.080901 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.080964 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.081109 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.081146 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260620 23:08:38.081202 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.077497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41673 (local address 127.0.0.1:41762)
0620 23:08:38.077646 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.077684 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.077749 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.078071 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.078073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.078075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:38.078117 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.078119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.079165 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.079166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.080240 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.080243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.080903 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.080904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.080996 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.081007 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.081093 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.081141 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":39,"threads_started":1}
I20260620 23:08:38.081425 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.077829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.1.32.65:40231)
0620 23:08:38.078027 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.078031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.078042 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:38.078080 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.078082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.078097 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.078175 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:38.078413 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.079408 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.080314 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.080966 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.081235 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.081293 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.081314 (+ 21us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.081351 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260620 23:08:38.081550 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.077625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.1.32.67:38523)
0620 23:08:38.078116 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.078122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.078130 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:38.078163 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.078165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.078177 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.078247 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:38.078370 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.079381 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.080510 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.081137 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.081345 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.081390 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.081408 (+ 18us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.081445 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":71,"threads_started":1}
I20260620 23:08:38.081508 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.077664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:40231 (local address 127.0.0.1:34466)
0620 23:08:38.077977 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.077985 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.078019 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.078289 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.078290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.078292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:38.078325 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.078326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.079515 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.079515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.080236 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.080238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.081056 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.081057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.081111 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.081114 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.081141 (+ 27us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.081184 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":43,"threads_started":1}
I20260620 23:08:38.081583 5690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.077559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:38523 (local address 127.0.0.1:42324)
0620 23:08:38.077836 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.077844 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.077888 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.078244 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.078245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.078246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:38.078278 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.078279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.079490 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.079491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.080436 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.080437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.081209 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.081210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.081257 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.081260 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.081284 (+ 24us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.081312 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":46,"threads_started":1}
I20260620 23:08:38.082353 5613 tablet_service.cc:1467] Processing CreateTablet for tablet a7d0fe530067459a919223c4a5643ec7 (DEFAULT_TABLE table=test-table [id=986320bab9eb45f29613243f9d401327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:38.082337 5480 tablet_service.cc:1467] Processing CreateTablet for tablet a7d0fe530067459a919223c4a5643ec7 (DEFAULT_TABLE table=test-table [id=986320bab9eb45f29613243f9d401327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:38.082686 5480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d0fe530067459a919223c4a5643ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:38.082686 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d0fe530067459a919223c4a5643ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:38.083263 5347 tablet_service.cc:1467] Processing CreateTablet for tablet a7d0fe530067459a919223c4a5643ec7 (DEFAULT_TABLE table=test-table [id=986320bab9eb45f29613243f9d401327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:38.083585 5347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d0fe530067459a919223c4a5643ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:38.085862 5696 tablet_bootstrap.cc:492] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Bootstrap starting.
I20260620 23:08:38.086282 5698 tablet_bootstrap.cc:492] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e: Bootstrap starting.
I20260620 23:08:38.086481 5697 tablet_bootstrap.cc:492] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Bootstrap starting.
I20260620 23:08:38.086824 5698 tablet_bootstrap.cc:654] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:38.087059 5697 tablet_bootstrap.cc:654] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:38.087365 5698 log.cc:824] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:38.087558 5697 log.cc:824] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:38.087985 5696 tablet_bootstrap.cc:654] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:38.088070 5698 tablet_bootstrap.cc:492] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e: No bootstrap required, opened a new log
I20260620 23:08:38.088145 5698 ts_tablet_manager.cc:1397] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:08:38.088253 5697 tablet_bootstrap.cc:492] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: No bootstrap required, opened a new log
I20260620 23:08:38.088325 5697 ts_tablet_manager.cc:1397] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:38.088604 5696 log.cc:824] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:38.089416 5696 tablet_bootstrap.cc:492] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: No bootstrap required, opened a new log
I20260620 23:08:38.089493 5696 ts_tablet_manager.cc:1397] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 23:08:38.090067 5698 raft_consensus.cc:348] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.090101 5697 raft_consensus.cc:348] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.090219 5697 raft_consensus.cc:374] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:38.090219 5698 raft_consensus.cc:374] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:38.090255 5698 raft_consensus.cc:729] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb1da398494425880f05351d6c5d01e, State: Initialized, Role: FOLLOWER
I20260620 23:08:38.090255 5697 raft_consensus.cc:729] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafa07fad5b447109b3063dd1f908c9e, State: Initialized, Role: FOLLOWER
I20260620 23:08:38.090366 5697 consensus_queue.cc:260] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.090359 5698 consensus_queue.cc:260] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.090620 5697 ts_tablet_manager.cc:1428] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:08:38.090620 5698 ts_tablet_manager.cc:1428] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 23:08:38.090626 5413 heartbeater.cc:503] Master 127.1.32.126:45333 was elected leader, sending a full tablet report...
I20260620 23:08:38.090732 5679 heartbeater.cc:503] Master 127.1.32.126:45333 was elected leader, sending a full tablet report...
I20260620 23:08:38.091104 5696 raft_consensus.cc:348] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.091233 5696 raft_consensus.cc:374] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:38.091290 5696 raft_consensus.cc:729] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a29fb187714276936c7e4e50210192, State: Initialized, Role: FOLLOWER
I20260620 23:08:38.091401 5696 consensus_queue.cc:260] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.091675 5696 ts_tablet_manager.cc:1428] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:08:38.091706 5546 heartbeater.cc:503] Master 127.1.32.126:45333 was elected leader, sending a full tablet report...
W20260620 23:08:38.133178 5547 tablet.cc:2369] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:38.170904 5702 raft_consensus.cc:482] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:38.171043 5702 raft_consensus.cc:504] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.171489 5702 leader_election.cc:288] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68a29fb187714276936c7e4e50210192 (127.1.32.66:41673), dafa07fad5b447109b3063dd1f908c9e (127.1.32.67:38523)
I20260620 23:08:38.174453 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.171762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57747 (local address 127.1.32.67:38523)
0620 23:08:38.171820 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.171822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.171831 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:38.171890 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.171890 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.171903 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.171976 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:38.172194 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.172711 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.173754 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.174106 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.174158 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.174166 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.174343 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.174386 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:08:38.174497 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.171812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57649 (local address 127.1.32.66:41673)
0620 23:08:38.171850 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.171851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.172052 (+ 201us) server_negotiation.cc:407] Connection header received
0620 23:08:38.172084 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.172085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.172116 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.172184 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:38.172330 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.172807 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.173753 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.174132 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.174229 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.174240 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.174395 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.174434 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:08:38.174497 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.171696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:38523 (local address 127.1.32.65:57747)
0620 23:08:38.171767 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.171780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.171829 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.171993 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.171993 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.171995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:38.172087 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.172089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.172800 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.172801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.173656 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.173659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.174189 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.174190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.174271 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.174275 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.174335 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.174391 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":57}
I20260620 23:08:38.174501 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.171707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41673 (local address 127.1.32.65:57649)
0620 23:08:38.171960 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.171970 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.172030 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.172195 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.172196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.172197 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:38.172262 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.172263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.172893 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.172894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.173660 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:38.173662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.174246 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.174247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:38.174297 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.174300 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.174372 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.174406 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":65,"thread_start_us":115,"threads_started":1}
I20260620 23:08:38.175002 5500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d0fe530067459a919223c4a5643ec7" candidate_uuid: "6cb1da398494425880f05351d6c5d01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68a29fb187714276936c7e4e50210192" is_pre_election: true
I20260620 23:08:38.174969 5633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d0fe530067459a919223c4a5643ec7" candidate_uuid: "6cb1da398494425880f05351d6c5d01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafa07fad5b447109b3063dd1f908c9e" is_pre_election: true
I20260620 23:08:38.175145 5633 raft_consensus.cc:2454] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cb1da398494425880f05351d6c5d01e in term 0.
I20260620 23:08:38.175145 5500 raft_consensus.cc:2454] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cb1da398494425880f05351d6c5d01e in term 0.
I20260620 23:08:38.175356 5301 leader_election.cc:302] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68a29fb187714276936c7e4e50210192, 6cb1da398494425880f05351d6c5d01e; no voters:
I20260620 23:08:38.175529 5702 raft_consensus.cc:2781] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:38.175581 5702 raft_consensus.cc:482] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:38.175598 5702 raft_consensus.cc:3037] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:38.176318 5702 raft_consensus.cc:504] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.176463 5702 leader_election.cc:288] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [CANDIDATE]: Term 1 election: Requested vote from peers 68a29fb187714276936c7e4e50210192 (127.1.32.66:41673), dafa07fad5b447109b3063dd1f908c9e (127.1.32.67:38523)
I20260620 23:08:38.176653 5633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d0fe530067459a919223c4a5643ec7" candidate_uuid: "6cb1da398494425880f05351d6c5d01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafa07fad5b447109b3063dd1f908c9e"
I20260620 23:08:38.176645 5500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d0fe530067459a919223c4a5643ec7" candidate_uuid: "6cb1da398494425880f05351d6c5d01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68a29fb187714276936c7e4e50210192"
I20260620 23:08:38.176728 5633 raft_consensus.cc:3037] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:38.176728 5500 raft_consensus.cc:3037] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:38.177649 5500 raft_consensus.cc:2454] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cb1da398494425880f05351d6c5d01e in term 1.
I20260620 23:08:38.177649 5633 raft_consensus.cc:2454] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cb1da398494425880f05351d6c5d01e in term 1.
I20260620 23:08:38.177820 5301 leader_election.cc:302] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68a29fb187714276936c7e4e50210192, 6cb1da398494425880f05351d6c5d01e; no voters:
I20260620 23:08:38.177922 5702 raft_consensus.cc:2781] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:38.178071 5702 raft_consensus.cc:686] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [term 1 LEADER]: Becoming Leader. State: Replica: 6cb1da398494425880f05351d6c5d01e, State: Running, Role: LEADER
I20260620 23:08:38.178170 5702 consensus_queue.cc:237] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } }
I20260620 23:08:38.179039 5222 catalog_manager.cc:5495] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cb1da398494425880f05351d6c5d01e (127.1.32.65). New cstate: current_term: 1 leader_uuid: "6cb1da398494425880f05351d6c5d01e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cb1da398494425880f05351d6c5d01e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 40231 } health_report { overall_health: HEALTHY } } }
I20260620 23:08:38.195068 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.192844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34472 (local address 127.1.32.65:40231)
0620 23:08:38.192897 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.192899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.192907 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:38.192969 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.192970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.192987 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.193043 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:38.193145 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.193554 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.194254 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:38.194421 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:38.194517 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:38.194671 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:38.194778 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:38.194839 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.194937 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.194957 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:08:38.196600 5633 raft_consensus.cc:1261] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 1 FOLLOWER]: Refusing update from remote peer 6cb1da398494425880f05351d6c5d01e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:38.196610 5500 raft_consensus.cc:1261] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 1 FOLLOWER]: Refusing update from remote peer 6cb1da398494425880f05351d6c5d01e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:38.196889 5706 consensus_queue.cc:1035] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafa07fad5b447109b3063dd1f908c9e" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:38.196988 5702 consensus_queue.cc:1035] T a7d0fe530067459a919223c4a5643ec7 P 6cb1da398494425880f05351d6c5d01e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68a29fb187714276936c7e4e50210192" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:38.199769 5711 mvcc.cc:205] Tried to move back new op lower bound from 7299059376930816000 to 7299059376858488832. Current Snapshot: MvccSnapshot[applied={T|T < 7299059376930816000}]
I20260620 23:08:38.200598 5710 mvcc.cc:205] Tried to move back new op lower bound from 7299059376930816000 to 7299059376858488832. Current Snapshot: MvccSnapshot[applied={T|T < 7299059376930816000}]
I20260620 23:08:38.200886 5222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:38.200976 5222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:38.201673 5222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84ee4cd41b284116ac26fa3b543aaade: Sending DeleteTablet for 3 replicas of tablet a7d0fe530067459a919223c4a5643ec7
I20260620 23:08:38.201988 5480 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d0fe530067459a919223c4a5643ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:38 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:41762
I20260620 23:08:38.202028 5347 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d0fe530067459a919223c4a5643ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:38 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:34466
I20260620 23:08:38.202195 5613 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d0fe530067459a919223c4a5643ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:38 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:42324
I20260620 23:08:38.202342 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5285
I20260620 23:08:38.202915 5719 tablet_replica.cc:330] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: stopping tablet replica
I20260620 23:08:38.203022 5719 raft_consensus.cc:2229] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:38.203110 5719 raft_consensus.cc:2258] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:38.203301 5721 tablet_replica.cc:330] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: stopping tablet replica
I20260620 23:08:38.203375 5721 raft_consensus.cc:2229] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:38.203454 5721 raft_consensus.cc:2258] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:38.203581 5719 ts_tablet_manager.cc:1905] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:38.204356 5721 ts_tablet_manager.cc:1905] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:38.205229 5719 ts_tablet_manager.cc:1918] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:38.205348 5719 log.cc:1196] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/a7d0fe530067459a919223c4a5643ec7
I20260620 23:08:38.205596 5719 ts_tablet_manager.cc:1939] T a7d0fe530067459a919223c4a5643ec7 P 68a29fb187714276936c7e4e50210192: Deleting consensus metadata
I20260620 23:08:38.205957 5208 catalog_manager.cc:4841] TS 68a29fb187714276936c7e4e50210192 (127.1.32.66:41673): tablet a7d0fe530067459a919223c4a5643ec7 (table test-table [id=986320bab9eb45f29613243f9d401327]) successfully deleted
I20260620 23:08:38.206209 5721 ts_tablet_manager.cc:1918] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:38.206293 5721 log.cc:1196] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/a7d0fe530067459a919223c4a5643ec7
I20260620 23:08:38.206539 5721 ts_tablet_manager.cc:1939] T a7d0fe530067459a919223c4a5643ec7 P dafa07fad5b447109b3063dd1f908c9e: Deleting consensus metadata
I20260620 23:08:38.206919 5207 catalog_manager.cc:4841] TS dafa07fad5b447109b3063dd1f908c9e (127.1.32.67:38523): tablet a7d0fe530067459a919223c4a5643ec7 (table test-table [id=986320bab9eb45f29613243f9d401327]) successfully deleted
W20260620 23:08:38.208546 5208 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:40231 (error 108)
I20260620 23:08:38.208689 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5418
I20260620 23:08:38.209347 5690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.209221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:40231 (local address 127.0.0.1:34480)
0620 23:08:38.209276 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.209312 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:40231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 23:08:38.209455 5208 catalog_manager.cc:4584] TS 6cb1da398494425880f05351d6c5d01e (127.1.32.65:40231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7d0fe530067459a919223c4a5643ec7: Network error: Client connection negotiation failed: client connection to 127.1.32.65:40231: connect: Connection refused (error 111)
I20260620 23:08:38.213284 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5551
I20260620 23:08:38.217598 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5192
2026-06-20T23:08:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1877 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T23:08:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:38Z Disabled control of system clock
I20260620 23:08:38.244812 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33275
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:43337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 23:08:38.319159 5726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:38.319334 5726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:38.319355 5726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:38.320356 5726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:38.320387 5726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:38.320399 5726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:38.320410 5726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:38.320421 5726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:38.321588 5726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:33275
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:38.321753 5726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:38.321949 5726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:38.326323 5732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:38.326333 5734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:38.326331 5731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:38.326551 5726 server_base.cc:1034] running on GCE node
I20260620 23:08:38.326792 5726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:38.327108 5726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:38.328258 5726 hybrid_clock.cc:648] HybridClock initialized: now 1781996918328219 us; error 38 us; skew 500 ppm
I20260620 23:08:38.329494 5726 webserver.cc:460] Webserver started at http://127.1.32.126:41743/ using document root <none> and password file <none>
I20260620 23:08:38.329855 5726 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:38.329918 5726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:38.330111 5726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:38.331256 5726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "5f58911155074ab7bd8787e5d43ed4d5"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "c84bc727173f8aa21dd45842df797e84"
server_key_iv: "f3d84b19f1afd23b34ab7301f2f24432"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.331629 5726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "5f58911155074ab7bd8787e5d43ed4d5"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "c84bc727173f8aa21dd45842df797e84"
server_key_iv: "f3d84b19f1afd23b34ab7301f2f24432"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.332856 5726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:08:38.333585 5740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.333746 5726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:38.333828 5726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "5f58911155074ab7bd8787e5d43ed4d5"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "c84bc727173f8aa21dd45842df797e84"
server_key_iv: "f3d84b19f1afd23b34ab7301f2f24432"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.333878 5726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:38.336953 5726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:38.337136 5726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:38.337224 5726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:38.340687 5726 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:33275
I20260620 23:08:38.340749 5792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:33275 every 8 connection(s)
I20260620 23:08:38.341046 5726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:38.341683 5793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:38.344312 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5: Bootstrap starting.
I20260620 23:08:38.344831 5793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:38.345106 5793 log.cc:824] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:38.345741 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5: No bootstrap required, opened a new log
I20260620 23:08:38.347295 5793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f58911155074ab7bd8787e5d43ed4d5" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33275 } }
I20260620 23:08:38.347436 5793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:38.347479 5793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f58911155074ab7bd8787e5d43ed4d5, State: Initialized, Role: FOLLOWER
I20260620 23:08:38.347631 5793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f58911155074ab7bd8787e5d43ed4d5" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33275 } }
I20260620 23:08:38.347707 5793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:38.347748 5793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:38.347792 5793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:38.348326 5793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f58911155074ab7bd8787e5d43ed4d5" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33275 } }
I20260620 23:08:38.348359 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5726
I20260620 23:08:38.348449 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:38.348439 5793 leader_election.cc:302] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f58911155074ab7bd8787e5d43ed4d5; no voters:
I20260620 23:08:38.348594 5793 leader_election.cc:288] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:38.348629 1153 external_mini_cluster.cc:1428] Setting key e261ed0d3d15a08837fe7268f55354ae
I20260620 23:08:38.348762 5797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:38.348853 5793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:38.348925 5797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [term 1 LEADER]: Becoming Leader. State: Replica: 5f58911155074ab7bd8787e5d43ed4d5, State: Running, Role: LEADER
I20260620 23:08:38.349031 5797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f58911155074ab7bd8787e5d43ed4d5" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33275 } }
I20260620 23:08:38.349346 5797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f58911155074ab7bd8787e5d43ed4d5. Latest consensus state: current_term: 1 leader_uuid: "5f58911155074ab7bd8787e5d43ed4d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f58911155074ab7bd8787e5d43ed4d5" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33275 } } }
I20260620 23:08:38.349431 5797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:38.349565 5798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f58911155074ab7bd8787e5d43ed4d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f58911155074ab7bd8787e5d43ed4d5" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33275 } } }
I20260620 23:08:38.349642 5798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:38.350179 5807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:38.350682 5807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:38.351635 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.349194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.1.32.126:33275)
0620 23:08:38.350189 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.350198 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.350233 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:08:38.350343 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.350349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.350387 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.350516 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.351190 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.351193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.351194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.351198 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.351241 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.351254 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.351333 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.351415 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.351470 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":76,"threads_started":1}
I20260620 23:08:38.352130 5807 catalog_manager.cc:1347] Generated new cluster ID: a805d19de82d43baa6abb878b226d5ad
I20260620 23:08:38.352176 5807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:38.355948 5807 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:38.356086 5807 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:38.361687 5807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5: Generated new TSK 0
I20260620 23:08:38.361835 5807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:38.366509 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:33275
--builtin_ntp_servers=127.1.32.84:43337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 23:08:38.440666 5817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:38.440884 5817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:38.440917 5817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:38.441962 5817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:38.442027 5817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:38.442070 5817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:38.443390 5817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:38.443640 5817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:38.443908 5817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:38.449247 5822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:38.449255 5823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:38.449290 5817 server_base.cc:1034] running on GCE node
W20260620 23:08:38.449524 5825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:38.449744 5817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:38.449998 5817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:38.451164 5817 hybrid_clock.cc:648] HybridClock initialized: now 1781996918451136 us; error 43 us; skew 500 ppm
I20260620 23:08:38.452440 5817 webserver.cc:460] Webserver started at http://127.1.32.65:33191/ using document root <none> and password file <none>
I20260620 23:08:38.452723 5817 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:38.452780 5817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:38.452903 5817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:38.454265 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "74e50f2a816f4fe58c198d66e9229b13"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "430084df65eb8754e82cbdc4e4de061e"
server_key_iv: "225ae3cf246fd8cb4aa1352a264798fb"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.454635 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "74e50f2a816f4fe58c198d66e9229b13"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "430084df65eb8754e82cbdc4e4de061e"
server_key_iv: "225ae3cf246fd8cb4aa1352a264798fb"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.456185 5817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:08:38.456916 5831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.457091 5817 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:38.457166 5817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "74e50f2a816f4fe58c198d66e9229b13"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "430084df65eb8754e82cbdc4e4de061e"
server_key_iv: "225ae3cf246fd8cb4aa1352a264798fb"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.457226 5817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:38.461285 5817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:38.461544 5817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:38.461647 5817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:38.461844 5817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:38.462108 5817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:38.462145 5817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.462168 5817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:38.462191 5817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.468753 5817 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:39249
I20260620 23:08:38.468778 5944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:39249 every 8 connection(s)
I20260620 23:08:38.469105 5817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:38.471108 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5817
I20260620 23:08:38.471215 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:38.471392 1153 external_mini_cluster.cc:1428] Setting key 692aaef54fc1ad7ec20697eecef42c34
I20260620 23:08:38.472215 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.469750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:49499 (local address 127.1.32.126:33275)
0620 23:08:38.469817 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.469819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.470112 (+ 293us) server_negotiation.cc:407] Connection header received
0620 23:08:38.470638 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.470639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.470645 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.470697 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.471914 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.471916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.471916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.471920 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.471934 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.471938 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.471982 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.472102 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.472119 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:08:38.472265 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:33275
--builtin_ntp_servers=127.1.32.84:43337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 23:08:38.472325 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.469742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:33275 (local address 127.1.32.65:49499)
0620 23:08:38.469939 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.469975 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.470528 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.470755 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.470758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.470768 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.471763 (+ 995us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.471763 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.471794 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.471823 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.472019 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.472023 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.472026 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.472090 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.472142 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260620 23:08:38.473052 5945 heartbeater.cc:347] Connected to a master server at 127.1.32.126:33275
I20260620 23:08:38.473142 5945 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:38.473256 5945 heartbeater.cc:511] Master 127.1.32.126:33275 requested a full tablet report, sending...
I20260620 23:08:38.473769 5757 ts_manager.cc:194] Registered new tserver with Master: 74e50f2a816f4fe58c198d66e9229b13 (127.1.32.65:39249)
W20260620 23:08:38.547953 5948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:38.548135 5948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:38.548154 5948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:38.549226 5948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:38.549263 5948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:38.549285 5948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:38.550542 5948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:38.550742 5948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:38.550971 5948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:38.555518 5953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:38.555506 5954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:38.555471 5956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:38.555737 5948 server_base.cc:1034] running on GCE node
I20260620 23:08:38.555994 5948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:38.556262 5948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:38.557410 5948 hybrid_clock.cc:648] HybridClock initialized: now 1781996918557405 us; error 34 us; skew 500 ppm
I20260620 23:08:38.558511 5948 webserver.cc:460] Webserver started at http://127.1.32.66:37479/ using document root <none> and password file <none>
I20260620 23:08:38.558729 5948 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:38.558791 5948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:38.558938 5948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:38.559888 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "69ecb715e812442a9097f8a93d629fbc"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "a4ea8a69b5032464350a5ed4dea241cc"
server_key_iv: "7fe3064b7cf746fbc70bb948c43e9131"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.560221 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "69ecb715e812442a9097f8a93d629fbc"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "a4ea8a69b5032464350a5ed4dea241cc"
server_key_iv: "7fe3064b7cf746fbc70bb948c43e9131"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.561543 5948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:38.562188 5962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.562356 5948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:38.562448 5948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "69ecb715e812442a9097f8a93d629fbc"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "a4ea8a69b5032464350a5ed4dea241cc"
server_key_iv: "7fe3064b7cf746fbc70bb948c43e9131"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.562533 5948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:38.565719 5948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:38.565956 5948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:38.566093 5948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:38.566285 5948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:38.566571 5948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:38.566623 5948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.566660 5948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:38.566687 5948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.573742 5948 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:44017
I20260620 23:08:38.573784 6075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:44017 every 8 connection(s)
I20260620 23:08:38.574052 5948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:38.576536 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.574674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:59699 (local address 127.1.32.126:33275)
0620 23:08:38.574758 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.574760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.574946 (+ 186us) server_negotiation.cc:407] Connection header received
0620 23:08:38.575358 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.575359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.575364 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.575415 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.576343 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.576344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.576345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.576348 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.576362 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.576366 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.576414 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.576474 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.576485 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 23:08:38.576789 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.574652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:33275 (local address 127.1.32.66:59699)
0620 23:08:38.574824 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.574859 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.575281 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.575458 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.575461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.575468 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.576227 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.576227 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.576251 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.576273 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.576411 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.576413 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.576416 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.576479 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.576526 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260620 23:08:38.577111 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 5948
I20260620 23:08:38.577196 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:38.577345 1153 external_mini_cluster.cc:1428] Setting key 8ec0a0439f290e4e1f2074fef4886be6
I20260620 23:08:38.577492 6076 heartbeater.cc:347] Connected to a master server at 127.1.32.126:33275
I20260620 23:08:38.577591 6076 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:38.577703 6076 heartbeater.cc:511] Master 127.1.32.126:33275 requested a full tablet report, sending...
I20260620 23:08:38.578153 5757 ts_manager.cc:194] Registered new tserver with Master: 69ecb715e812442a9097f8a93d629fbc (127.1.32.66:44017)
I20260620 23:08:38.578336 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:33275
--builtin_ntp_servers=127.1.32.84:43337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 23:08:38.655181 6079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:38.655367 6079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:38.655388 6079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:38.656507 6079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:38.656554 6079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:38.656579 6079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:38.657907 6079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:38.658142 6079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:38.658411 6079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:38.663309 6085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:38.663301 6087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:38.663297 6084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:38.663381 6079 server_base.cc:1034] running on GCE node
I20260620 23:08:38.663718 6079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:38.663951 6079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:38.665107 6079 hybrid_clock.cc:648] HybridClock initialized: now 1781996918665100 us; error 39 us; skew 500 ppm
I20260620 23:08:38.666376 6079 webserver.cc:460] Webserver started at http://127.1.32.67:33855/ using document root <none> and password file <none>
I20260620 23:08:38.666612 6079 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:38.666683 6079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:38.666810 6079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:38.667851 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "03be0165234b4a16b8be1d1dcb49553d"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "f8a34decb9da22659098e31c226bc543"
server_key_iv: "26a7b47374ba0f3e08248abcc5af105d"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.668190 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "03be0165234b4a16b8be1d1dcb49553d"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "f8a34decb9da22659098e31c226bc543"
server_key_iv: "26a7b47374ba0f3e08248abcc5af105d"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.669374 6079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:38.670204 6093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.670387 6079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:38.670480 6079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "03be0165234b4a16b8be1d1dcb49553d"
format_stamp: "Formatted at 2026-06-20 23:08:38 on dist-test-slave-fzjm"
server_key: "f8a34decb9da22659098e31c226bc543"
server_key_iv: "26a7b47374ba0f3e08248abcc5af105d"
server_key_version: "encryptionkey@0"
I20260620 23:08:38.670579 6079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:38.673913 6079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:38.674142 6079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:38.674264 6079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:38.674461 6079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:38.674762 6079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:38.674811 6079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.674862 6079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:38.674904 6079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:38.680289 6079 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39283
I20260620 23:08:38.680327 6206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39283 every 8 connection(s)
I20260620 23:08:38.680601 6079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:08:38.682713 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6079
I20260620 23:08:38.682838 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:38.683113 1153 external_mini_cluster.cc:1428] Setting key d28967c693f0084fbab2c9360841ef69
I20260620 23:08:38.683526 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.681372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:44313 (local address 127.1.32.126:33275)
0620 23:08:38.681414 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.681416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.681510 (+ 94us) server_negotiation.cc:407] Connection header received
0620 23:08:38.681873 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.681874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.681877 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.681929 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.683280 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.683281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.683282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.683292 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.683303 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.683307 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.683334 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.683445 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.683460 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:38.683704 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.681221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:33275 (local address 127.1.32.67:44313)
0620 23:08:38.681384 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.681419 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.681802 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.681971 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.681973 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.681980 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.683089 (+ 1109us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.683090 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.683124 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.683157 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.683366 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.683370 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.683372 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.683452 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.683515 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260620 23:08:38.684108 6207 heartbeater.cc:347] Connected to a master server at 127.1.32.126:33275
I20260620 23:08:38.684175 6207 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:38.684350 6207 heartbeater.cc:511] Master 127.1.32.126:33275 requested a full tablet report, sending...
I20260620 23:08:38.684688 5757 ts_manager.cc:194] Registered new tserver with Master: 03be0165234b4a16b8be1d1dcb49553d (127.1.32.67:39283)
I20260620 23:08:38.694833 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:08:38.696974 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.696122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.1.32.126:33275)
0620 23:08:38.696179 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.696182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.696251 (+ 69us) server_negotiation.cc:407] Connection header received
0620 23:08:38.696306 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.696307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.696310 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.696353 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.696829 (+ 476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.696830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.696830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.696833 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.696845 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.696848 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.696892 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.696922 (+ 30us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.696933 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:08:38.698378 5757 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:38.699842 5757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:38.703279 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.701685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:39249 (local address 127.0.0.1:44112)
0620 23:08:38.701910 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.701957 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.702014 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.702364 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.702365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.702366 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.702809 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.702810 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.702816 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.702829 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.703085 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.703090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.703092 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.703193 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.703205 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":49,"threads_started":1}
I20260620 23:08:38.703423 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.701968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.1.32.66:44017)
0620 23:08:38.702140 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.702143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.702152 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:38.702176 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.702178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.702186 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.702263 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.703092 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.703102 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.703102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.703107 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.703136 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.703167 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.703218 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.703326 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.703340 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":61,"threads_started":1}
I20260620 23:08:38.703435 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.701733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39283 (local address 127.0.0.1:51400)
0620 23:08:38.702088 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.702094 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.702133 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.702274 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.702278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.702283 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.702679 (+ 396us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.702679 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.702693 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.702710 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.703327 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.703328 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.703329 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.703366 (+ 37us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.703379 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260620 23:08:38.703600 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.701837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.1.32.65:39249)
0620 23:08:38.702224 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.702227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.702233 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:08:38.702272 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.702274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.702281 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.702470 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.702934 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.702940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.702940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.702943 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.702972 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.703005 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.703477 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.703498 (+ 21us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.703511 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":95,"threads_started":1}
I20260620 23:08:38.703739 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.701733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:44017 (local address 127.0.0.1:41316)
0620 23:08:38.702041 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.702049 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.702082 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.702461 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.702462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.702464 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.702997 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.702997 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.703003 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.703012 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.703239 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.703240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.703240 (+ 0us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.703277 (+ 37us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.703286 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":112,"threads_started":1}
I20260620 23:08:38.703853 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.701952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51400 (local address 127.1.32.67:39283)
0620 23:08:38.702079 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.702083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.702153 (+ 70us) server_negotiation.cc:407] Connection header received
0620 23:08:38.702180 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.702182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.702188 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.702263 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.703164 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.703167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.703167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.703170 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.703200 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.703213 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.703330 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.703770 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.703786 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":53,"threads_started":1}
I20260620 23:08:38.704327 6141 tablet_service.cc:1467] Processing CreateTablet for tablet 099db0ae5b5d4a5aa81c874a59fa6a73 (DEFAULT_TABLE table=test-table [id=3a57c2710c3246f58ae3dc309b886bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:38.704281 5879 tablet_service.cc:1467] Processing CreateTablet for tablet 099db0ae5b5d4a5aa81c874a59fa6a73 (DEFAULT_TABLE table=test-table [id=3a57c2710c3246f58ae3dc309b886bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:38.704293 6009 tablet_service.cc:1467] Processing CreateTablet for tablet 099db0ae5b5d4a5aa81c874a59fa6a73 (DEFAULT_TABLE table=test-table [id=3a57c2710c3246f58ae3dc309b886bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:38.704617 6141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099db0ae5b5d4a5aa81c874a59fa6a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:38.704617 5879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099db0ae5b5d4a5aa81c874a59fa6a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:38.704617 6009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099db0ae5b5d4a5aa81c874a59fa6a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:38.707261 6221 tablet_bootstrap.cc:492] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Bootstrap starting.
I20260620 23:08:38.707834 6221 tablet_bootstrap.cc:654] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:38.707847 6222 tablet_bootstrap.cc:492] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: Bootstrap starting.
I20260620 23:08:38.708097 6221 log.cc:824] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:38.708401 6223 tablet_bootstrap.cc:492] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Bootstrap starting.
I20260620 23:08:38.708427 6222 tablet_bootstrap.cc:654] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:38.708717 6222 log.cc:824] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:38.708804 6221 tablet_bootstrap.cc:492] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: No bootstrap required, opened a new log
I20260620 23:08:38.708878 6221 ts_tablet_manager.cc:1397] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:38.708940 6223 tablet_bootstrap.cc:654] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:38.709291 6223 log.cc:824] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:38.709440 6222 tablet_bootstrap.cc:492] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: No bootstrap required, opened a new log
I20260620 23:08:38.709494 6222 ts_tablet_manager.cc:1397] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:38.710018 6223 tablet_bootstrap.cc:492] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: No bootstrap required, opened a new log
I20260620 23:08:38.710073 6223 ts_tablet_manager.cc:1397] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:38.710857 6221 raft_consensus.cc:348] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.711047 6221 raft_consensus.cc:374] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:38.711088 6221 raft_consensus.cc:729] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03be0165234b4a16b8be1d1dcb49553d, State: Initialized, Role: FOLLOWER
I20260620 23:08:38.711200 6221 consensus_queue.cc:260] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.711440 6221 ts_tablet_manager.cc:1428] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:38.711565 6207 heartbeater.cc:503] Master 127.1.32.126:33275 was elected leader, sending a full tablet report...
I20260620 23:08:38.711874 6222 raft_consensus.cc:348] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.712049 6222 raft_consensus.cc:374] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:38.712093 6222 raft_consensus.cc:729] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e50f2a816f4fe58c198d66e9229b13, State: Initialized, Role: FOLLOWER
I20260620 23:08:38.712208 6222 consensus_queue.cc:260] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.712426 6222 ts_tablet_manager.cc:1428] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:38.712460 6223 raft_consensus.cc:348] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.712558 5945 heartbeater.cc:503] Master 127.1.32.126:33275 was elected leader, sending a full tablet report...
I20260620 23:08:38.712569 6223 raft_consensus.cc:374] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:38.712601 6223 raft_consensus.cc:729] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ecb715e812442a9097f8a93d629fbc, State: Initialized, Role: FOLLOWER
I20260620 23:08:38.712695 6223 consensus_queue.cc:260] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.712909 6223 ts_tablet_manager.cc:1428] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:38.713011 6076 heartbeater.cc:503] Master 127.1.32.126:33275 was elected leader, sending a full tablet report...
W20260620 23:08:38.719704 5946 tablet.cc:2369] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:08:38.824584 6077 tablet.cc:2369] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:38.911899 6227 raft_consensus.cc:482] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:38.912034 6227 raft_consensus.cc:504] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.912370 6227 leader_election.cc:288] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e50f2a816f4fe58c198d66e9229b13 (127.1.32.65:39249), 69ecb715e812442a9097f8a93d629fbc (127.1.32.66:44017)
I20260620 23:08:38.913863 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.912654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:40553 (local address 127.1.32.65:39249)
0620 23:08:38.912727 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.912728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.912739 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:38.912765 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.912766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.912771 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.912874 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.913592 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.913594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.913594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.913606 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.913620 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.913624 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.913689 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.913744 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.913757 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:08:38.913863 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.912581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:39249 (local address 127.1.32.67:40553)
0620 23:08:38.912629 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.912641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.912703 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.912883 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.912885 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.912887 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.913457 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.913457 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.913464 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.913475 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.913684 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.913686 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.913686 (+ 0us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.913742 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.913757 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 23:08:38.913967 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.912653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:40791 (local address 127.1.32.66:44017)
0620 23:08:38.912722 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.912724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.912994 (+ 270us) server_negotiation.cc:407] Connection header received
0620 23:08:38.913023 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.913025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.913031 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.913095 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.913711 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.913712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.913713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.913718 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.913733 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.913738 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.913809 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.913869 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.913881 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:08:38.913996 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.912581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:44017 (local address 127.1.32.67:40791)
0620 23:08:38.912903 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.912912 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:38.912963 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:38.913117 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:38.913117 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:38.913120 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:38.913608 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:08:38.913609 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:38.913616 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:08:38.913626 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:38.913800 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:38.913800 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:38.913801 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:08:38.913859 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:08:38.913872 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":171,"thread_start_us":105,"threads_started":1}
I20260620 23:08:38.914274 5899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099db0ae5b5d4a5aa81c874a59fa6a73" candidate_uuid: "03be0165234b4a16b8be1d1dcb49553d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e50f2a816f4fe58c198d66e9229b13" is_pre_election: true
I20260620 23:08:38.914291 6029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099db0ae5b5d4a5aa81c874a59fa6a73" candidate_uuid: "03be0165234b4a16b8be1d1dcb49553d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69ecb715e812442a9097f8a93d629fbc" is_pre_election: true
I20260620 23:08:38.914448 6029 raft_consensus.cc:2454] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03be0165234b4a16b8be1d1dcb49553d in term 0.
I20260620 23:08:38.914450 5899 raft_consensus.cc:2454] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03be0165234b4a16b8be1d1dcb49553d in term 0.
I20260620 23:08:38.914708 6095 leader_election.cc:302] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be0165234b4a16b8be1d1dcb49553d, 74e50f2a816f4fe58c198d66e9229b13; no voters:
I20260620 23:08:38.914846 6227 raft_consensus.cc:2781] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:38.914930 6227 raft_consensus.cc:482] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:38.914968 6227 raft_consensus.cc:3037] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:38.915686 6227 raft_consensus.cc:504] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.915838 6227 leader_election.cc:288] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [CANDIDATE]: Term 1 election: Requested vote from peers 74e50f2a816f4fe58c198d66e9229b13 (127.1.32.65:39249), 69ecb715e812442a9097f8a93d629fbc (127.1.32.66:44017)
I20260620 23:08:38.915972 5899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099db0ae5b5d4a5aa81c874a59fa6a73" candidate_uuid: "03be0165234b4a16b8be1d1dcb49553d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e50f2a816f4fe58c198d66e9229b13"
I20260620 23:08:38.916049 5899 raft_consensus.cc:3037] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:38.916006 6029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099db0ae5b5d4a5aa81c874a59fa6a73" candidate_uuid: "03be0165234b4a16b8be1d1dcb49553d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69ecb715e812442a9097f8a93d629fbc"
I20260620 23:08:38.916108 6029 raft_consensus.cc:3037] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:38.916672 5899 raft_consensus.cc:2454] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03be0165234b4a16b8be1d1dcb49553d in term 1.
I20260620 23:08:38.916832 6095 leader_election.cc:302] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be0165234b4a16b8be1d1dcb49553d, 74e50f2a816f4fe58c198d66e9229b13; no voters:
I20260620 23:08:38.916926 6029 raft_consensus.cc:2454] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03be0165234b4a16b8be1d1dcb49553d in term 1.
I20260620 23:08:38.916939 6227 raft_consensus.cc:2781] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:38.917071 6227 raft_consensus.cc:686] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 1 LEADER]: Becoming Leader. State: Replica: 03be0165234b4a16b8be1d1dcb49553d, State: Running, Role: LEADER
I20260620 23:08:38.917181 6227 consensus_queue.cc:237] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } }
I20260620 23:08:38.918006 5756 catalog_manager.cc:5495] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d reported cstate change: term changed from 0 to 1, leader changed from <none> to 03be0165234b4a16b8be1d1dcb49553d (127.1.32.67). New cstate: current_term: 1 leader_uuid: "03be0165234b4a16b8be1d1dcb49553d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03be0165234b4a16b8be1d1dcb49553d" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 } health_report { overall_health: UNKNOWN } } }
W20260620 23:08:38.931034 6208 tablet.cc:2369] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:38.932070 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.931215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.1.32.67:39283)
0620 23:08:38.931263 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:38.931265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:38.931273 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:38.931349 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:38.931349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:38.931360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:38.931406 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:38.931901 (+ 495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:38.931903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:38.931903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:38.931906 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:38.931918 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:38.931922 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:38.931978 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:38.932017 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 23:08:38.932027 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:08:38.933717 5899 raft_consensus.cc:1261] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 1 FOLLOWER]: Refusing update from remote peer 03be0165234b4a16b8be1d1dcb49553d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:38.934033 6029 raft_consensus.cc:1261] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 1 FOLLOWER]: Refusing update from remote peer 03be0165234b4a16b8be1d1dcb49553d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:38.934019 6231 consensus_queue.cc:1035] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e50f2a816f4fe58c198d66e9229b13" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 39249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:38.934306 6231 consensus_queue.cc:1035] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [LEADER]: Connected to new peer: Peer: permanent_uuid: "69ecb715e812442a9097f8a93d629fbc" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 44017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:38.937083 6236 mvcc.cc:205] Tried to move back new op lower bound from 7299059379950112768 to 7299059379885514752. Current Snapshot: MvccSnapshot[applied={T|T < 7299059379950112768}]
I20260620 23:08:38.937601 6234 mvcc.cc:205] Tried to move back new op lower bound from 7299059379950112768 to 7299059379885514752. Current Snapshot: MvccSnapshot[applied={T|T < 7299059379950112768 or (T in {7299059379950112768})}]
I20260620 23:08:38.937693 6235 mvcc.cc:205] Tried to move back new op lower bound from 7299059379950112768 to 7299059379885514752. Current Snapshot: MvccSnapshot[applied={T|T < 7299059379950112768 or (T in {7299059379950112768})}]
I20260620 23:08:38.938594 5757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:38.938694 5757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:38.939452 5757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f58911155074ab7bd8787e5d43ed4d5: Sending DeleteTablet for 3 replicas of tablet 099db0ae5b5d4a5aa81c874a59fa6a73
I20260620 23:08:38.939807 6141 tablet_service.cc:1514] Processing DeleteTablet for tablet 099db0ae5b5d4a5aa81c874a59fa6a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:38 UTC) from {username='slave'} at 127.0.0.1:51400
I20260620 23:08:38.939822 6009 tablet_service.cc:1514] Processing DeleteTablet for tablet 099db0ae5b5d4a5aa81c874a59fa6a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:38 UTC) from {username='slave'} at 127.0.0.1:41316
I20260620 23:08:38.940037 5879 tablet_service.cc:1514] Processing DeleteTablet for tablet 099db0ae5b5d4a5aa81c874a59fa6a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:38 UTC) from {username='slave'} at 127.0.0.1:44112
I20260620 23:08:38.940251 6246 tablet_replica.cc:330] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13: stopping tablet replica
I20260620 23:08:38.940464 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5817
I20260620 23:08:38.940507 6246 raft_consensus.cc:2229] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:38.940698 6246 raft_consensus.cc:2258] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 74e50f2a816f4fe58c198d66e9229b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:38.940410 6245 tablet_replica.cc:330] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: stopping tablet replica
I20260620 23:08:38.941464 6245 raft_consensus.cc:2229] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:38.941555 6245 raft_consensus.cc:2258] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:38.941721 6244 tablet_replica.cc:330] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: stopping tablet replica
I20260620 23:08:38.941812 6244 raft_consensus.cc:2229] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:38.941910 6244 raft_consensus.cc:2258] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:38.942109 6245 ts_tablet_manager.cc:1905] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:38.942338 6244 ts_tablet_manager.cc:1905] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:38.944013 6244 ts_tablet_manager.cc:1918] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:38.944013 6245 ts_tablet_manager.cc:1918] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:38.944090 6244 log.cc:1196] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/099db0ae5b5d4a5aa81c874a59fa6a73
I20260620 23:08:38.944198 6245 log.cc:1196] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/099db0ae5b5d4a5aa81c874a59fa6a73
I20260620 23:08:38.944362 6244 ts_tablet_manager.cc:1939] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 69ecb715e812442a9097f8a93d629fbc: Deleting consensus metadata
I20260620 23:08:38.944593 6245 ts_tablet_manager.cc:1939] T 099db0ae5b5d4a5aa81c874a59fa6a73 P 03be0165234b4a16b8be1d1dcb49553d: Deleting consensus metadata
I20260620 23:08:38.944880 5741 catalog_manager.cc:4841] TS 69ecb715e812442a9097f8a93d629fbc (127.1.32.66:44017): tablet 099db0ae5b5d4a5aa81c874a59fa6a73 (table test-table [id=3a57c2710c3246f58ae3dc309b886bbb]) successfully deleted
I20260620 23:08:38.944947 5743 catalog_manager.cc:4841] TS 03be0165234b4a16b8be1d1dcb49553d (127.1.32.67:39283): tablet 099db0ae5b5d4a5aa81c874a59fa6a73 (table test-table [id=3a57c2710c3246f58ae3dc309b886bbb]) successfully deleted
W20260620 23:08:38.947065 5742 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:39249 (error 108)
I20260620 23:08:38.947655 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:38.947529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:39249 (local address 127.0.0.1:44128)
0620 23:08:38.947582 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:38.947620 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:39249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:08:38.947671 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5948
W20260620 23:08:38.949002 5742 catalog_manager.cc:4584] TS 74e50f2a816f4fe58c198d66e9229b13 (127.1.32.65:39249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 099db0ae5b5d4a5aa81c874a59fa6a73: Network error: Client connection negotiation failed: client connection to 127.1.32.65:39249: connect: Connection refused (error 111)
I20260620 23:08:38.952975 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6079
I20260620 23:08:38.956817 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 5726
2026-06-20T23:08:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (737 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 23:08:38.975708 1153 mini_oidc.cc:152] Starting JWKS server
I20260620 23:08:38.976755 1153 webserver.cc:460] Webserver started at https://127.0.0.1:42593/ using document root <none> and password file <none>
I20260620 23:08:38.976874 1153 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 23:08:38.977048 1153 webserver.cc:460] Webserver started at http://127.0.0.1:39415/ using document root <none> and password file <none>
2026-06-20T23:08:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:38Z Disabled control of system clock
I20260620 23:08:38.985316 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33309
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:33309
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42593/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/testchainca.cert with env {}
W20260620 23:08:39.060427 6253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:39.060659 6253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:39.060684 6253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:39.061775 6253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:39.061807 6253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:39.061820 6253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:39.061831 6253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:39.061843 6253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:08:39.061854 6253 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:08:39.061865 6253 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42593/jwks.json
I20260620 23:08:39.063265 6253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:33309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42593/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:39.063443 6253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:39.063598 6253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:39.068039 6259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:39.068032 6261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:39.068050 6253 server_base.cc:1034] running on GCE node
W20260620 23:08:39.068066 6258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:39.068574 6253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:39.068836 6253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:39.069995 6253 hybrid_clock.cc:648] HybridClock initialized: now 1781996919069975 us; error 46 us; skew 500 ppm
I20260620 23:08:39.071192 6253 webserver.cc:460] Webserver started at http://127.1.32.126:45687/ using document root <none> and password file <none>
I20260620 23:08:39.071435 6253 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:39.071519 6253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:39.071655 6253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:39.072573 6253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "209a4ef42ba24ad386a939baab6bff97"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "3d8ca023fdd8858cb329231c8d30976a"
server_key_iv: "f8f5426164c795446651fa27d8a3887d"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.072906 6253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "209a4ef42ba24ad386a939baab6bff97"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "3d8ca023fdd8858cb329231c8d30976a"
server_key_iv: "f8f5426164c795446651fa27d8a3887d"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.074077 6253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:39.074693 6267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:39.074864 6253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:39.074978 6253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "209a4ef42ba24ad386a939baab6bff97"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "3d8ca023fdd8858cb329231c8d30976a"
server_key_iv: "f8f5426164c795446651fa27d8a3887d"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.075058 6253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:39.113682 6253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:39.113978 6253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:39.114128 6253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:39.117972 6253 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:33309
I20260620 23:08:39.118006 6319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:33309 every 8 connection(s)
I20260620 23:08:39.118321 6253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:39.119282 6320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:39.119848 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6253
I20260620 23:08:39.119939 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:39.120103 1153 external_mini_cluster.cc:1428] Setting key 17a68a09d7f2afa699030936a71abd40
I20260620 23:08:39.122265 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97: Bootstrap starting.
I20260620 23:08:39.122949 6320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:39.123222 6320 log.cc:824] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:39.123914 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97: No bootstrap required, opened a new log
I20260620 23:08:39.126082 6323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.120582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.1.32.126:33309)
0620 23:08:39.120794 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.120802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.120833 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:08:39.121444 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.121454 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.121546 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.121680 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:39.121832 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.123096 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.123975 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.124575 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.124706 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.125304 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:39.125308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:39.125308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:39.125522 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:39.125563 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:39.125579 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:39.125680 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:39.125746 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 23:08:39.125840 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260620 23:08:39.126281 6320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209a4ef42ba24ad386a939baab6bff97" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33309 } }
I20260620 23:08:39.126430 6320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:39.126470 6320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 209a4ef42ba24ad386a939baab6bff97, State: Initialized, Role: FOLLOWER
I20260620 23:08:39.126608 6320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209a4ef42ba24ad386a939baab6bff97" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33309 } }
I20260620 23:08:39.126700 6320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:39.126744 6320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:39.126788 6320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:39.127283 6320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209a4ef42ba24ad386a939baab6bff97" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33309 } }
I20260620 23:08:39.127395 6320 leader_election.cc:302] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 209a4ef42ba24ad386a939baab6bff97; no voters:
I20260620 23:08:39.127547 6320 leader_election.cc:288] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:39.127588 6325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:39.127626 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:39.127686 6325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [term 1 LEADER]: Becoming Leader. State: Replica: 209a4ef42ba24ad386a939baab6bff97, State: Running, Role: LEADER
I20260620 23:08:39.127754 6325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209a4ef42ba24ad386a939baab6bff97" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33309 } }
I20260620 23:08:39.127802 6320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:39.128059 6325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 209a4ef42ba24ad386a939baab6bff97. Latest consensus state: current_term: 1 leader_uuid: "209a4ef42ba24ad386a939baab6bff97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209a4ef42ba24ad386a939baab6bff97" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33309 } } }
I20260620 23:08:39.128069 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "209a4ef42ba24ad386a939baab6bff97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209a4ef42ba24ad386a939baab6bff97" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33309 } } }
I20260620 23:08:39.128129 6325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:39.128181 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:39.128350 6330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:39.129004 6330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:39.130419 6330 catalog_manager.cc:1347] Generated new cluster ID: e2129fd827d14e5e8514ca88876d520a
I20260620 23:08:39.130470 6330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:39.140041 6330 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:39.140477 6330 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:39.145766 6330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 209a4ef42ba24ad386a939baab6bff97: Generated new TSK 0
I20260620 23:08:39.145939 6330 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 23:08:39.174448 6350 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 23:08:39.174662 6350 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 23:08:39.174700 6323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.153274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.1.32.126:33309)
0620 23:08:39.153316 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.153317 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.153447 (+ 130us) server_negotiation.cc:407] Connection header received
0620 23:08:39.153543 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.153543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.153551 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.153603 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:08:39.153745 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.154192 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.155267 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.155450 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.155502 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.155721 (+ 219us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:08:39.174431 (+ 18710us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 23:08:39.174592 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 23:08:39.174791 6323 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 23:08:39.175047 6349 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:39.153200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:33309 (local address 127.0.0.1:43356)
0620 23:08:39.153379 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:39.153390 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:39.153455 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:39.153624 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:39.153625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:39.153627 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:08:39.153687 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.153689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.154259 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.154260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.155195 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.155197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.155536 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.155538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.155602 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.155623 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:08:39.174556 (+ 18933us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 23:08:39.174692 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.32.126:33309: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.32.126:33309: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 23:08:39.175511 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6253
2026-06-20T23:08:39Z chronyd exiting
I20260620 23:08:39.211648 1153 test_util.cc:175] -----------------------------------------------
I20260620 23:08:39.211709 1153 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996897900784-1153-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 23:08:39.212422 1153 mini_oidc.cc:152] Starting JWKS server
I20260620 23:08:39.212992 1153 webserver.cc:460] Webserver started at https://127.0.0.1:42799/ using document root <none> and password file <none>
I20260620 23:08:39.213029 1153 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 23:08:39.213160 1153 webserver.cc:460] Webserver started at http://127.0.0.1:39755/ using document root <none> and password file <none>
2026-06-20T23:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:39Z Disabled control of system clock
I20260620 23:08:39.220477 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40899
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:37123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:40899
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42799/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/testchainca.cert with env {}
W20260620 23:08:39.297160 6356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:39.297400 6356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:39.297435 6356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:39.298557 6356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:39.298619 6356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:39.298641 6356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:39.298671 6356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:39.298692 6356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:08:39.298717 6356 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:08:39.298741 6356 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42799/jwks.json
I20260620 23:08:39.300271 6356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:40899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42799/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:39.300529 6356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:39.300724 6356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:39.305315 6362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:39.305362 6364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:39.305326 6361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:39.305541 6356 server_base.cc:1034] running on GCE node
I20260620 23:08:39.305866 6356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:39.306156 6356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:39.307322 6356 hybrid_clock.cc:648] HybridClock initialized: now 1781996919307301 us; error 33 us; skew 500 ppm
I20260620 23:08:39.308645 6356 webserver.cc:460] Webserver started at http://127.1.32.126:41433/ using document root <none> and password file <none>
I20260620 23:08:39.308897 6356 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:39.308964 6356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:39.309119 6356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:39.310190 6356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "c3de820c930640b68de93f99616455c9"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "f4a4a650230c79b9cf67852ba5c31e18"
server_key_iv: "ba4db784c95b7632bedc617b30a5aa7c"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.310619 6356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "c3de820c930640b68de93f99616455c9"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "f4a4a650230c79b9cf67852ba5c31e18"
server_key_iv: "ba4db784c95b7632bedc617b30a5aa7c"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.311980 6356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:39.312777 6370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:39.312959 6356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:39.313060 6356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "c3de820c930640b68de93f99616455c9"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "f4a4a650230c79b9cf67852ba5c31e18"
server_key_iv: "ba4db784c95b7632bedc617b30a5aa7c"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.313135 6356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:39.321935 6356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:39.322268 6356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:39.322446 6356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:39.326385 6356 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:40899
I20260620 23:08:39.326409 6422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:40899 every 8 connection(s)
I20260620 23:08:39.326733 6356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:39.327706 6423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:39.329808 6423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9: Bootstrap starting.
I20260620 23:08:39.330300 6423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:39.330535 6423 log.cc:824] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:39.331220 6423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9: No bootstrap required, opened a new log
I20260620 23:08:39.332687 6423 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de820c930640b68de93f99616455c9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40899 } }
I20260620 23:08:39.332813 6423 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:39.332834 6423 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3de820c930640b68de93f99616455c9, State: Initialized, Role: FOLLOWER
I20260620 23:08:39.332963 6423 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de820c930640b68de93f99616455c9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40899 } }
I20260620 23:08:39.333038 6423 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:39.333060 6423 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:39.333086 6423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:39.333531 6423 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de820c930640b68de93f99616455c9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40899 } }
I20260620 23:08:39.333607 6423 leader_election.cc:302] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3de820c930640b68de93f99616455c9; no voters:
I20260620 23:08:39.333721 6423 leader_election.cc:288] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:39.333803 6426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:39.333978 6426 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [term 1 LEADER]: Becoming Leader. State: Replica: c3de820c930640b68de93f99616455c9, State: Running, Role: LEADER
I20260620 23:08:39.333992 6423 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:39.334096 6426 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de820c930640b68de93f99616455c9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40899 } }
I20260620 23:08:39.334337 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6356
I20260620 23:08:39.334445 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:39.334724 1153 external_mini_cluster.cc:1428] Setting key de8e8c7a09265393e54daf018fe93432
I20260620 23:08:39.335227 6427 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3de820c930640b68de93f99616455c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de820c930640b68de93f99616455c9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40899 } } }
I20260620 23:08:39.335321 6427 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:39.335283 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3de820c930640b68de93f99616455c9. Latest consensus state: current_term: 1 leader_uuid: "c3de820c930640b68de93f99616455c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de820c930640b68de93f99616455c9" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40899 } } }
I20260620 23:08:39.335369 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9 [sys.catalog]: This master's current role is: LEADER
W20260620 23:08:39.336056 6441 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:39.336112 6441 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:39.336162 6443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:39.336356 6443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:39.337741 6443 catalog_manager.cc:1347] Generated new cluster ID: d272b9df5d10457f8db5a929e90e1993
I20260620 23:08:39.337795 6443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:39.341920 6444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.335645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.1.32.126:40899)
0620 23:08:39.336085 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.336092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.336844 (+ 752us) server_negotiation.cc:407] Connection header received
0620 23:08:39.336934 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.336939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.336996 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.337144 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:39.337369 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.338719 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.339818 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.340451 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.340573 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.341273 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:39.341277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:39.341277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:39.341462 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:39.341499 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:39.341512 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:39.341577 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:39.341642 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 23:08:39.341709 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":73,"threads_started":1}
I20260620 23:08:39.342789 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:39.347244 6443 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:39.347718 6443 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:39.351028 6443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3de820c930640b68de93f99616455c9: Generated new TSK 0
I20260620 23:08:39.351183 6443 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 23:08:39.372682 6453 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 23:08:39.372781 6453 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 23:08:39.373096 6444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.358774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53530 (local address 127.1.32.126:40899)
0620 23:08:39.358815 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.358817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.358907 (+ 90us) server_negotiation.cc:407] Connection header received
0620 23:08:39.358992 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.358993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.359003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.359075 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:08:39.359174 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.359888 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.360551 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.360991 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.361061 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.361245 (+ 184us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:08:39.372868 (+ 11623us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 23:08:39.373026 (+ 158us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53530: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 23:08:39.373153 6444 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53530: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 23:08:39.373190 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:39.358709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40899 (local address 127.0.0.1:53530)
0620 23:08:39.358833 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:39.358842 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:39.358930 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:39.359079 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:39.359080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:39.359081 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:08:39.359123 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.359124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.359968 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.359969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.360478 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.360480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.361094 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.361095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.361143 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.361148 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:08:39.372984 (+ 11836us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 23:08:39.373090 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.32.126:40899: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
I20260620 23:08:39.373447 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6356
2026-06-20T23:08:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (198 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 23:08:39.411080 1153 mini_oidc.cc:152] Starting JWKS server
I20260620 23:08:39.411747 1153 webserver.cc:460] Webserver started at https://127.0.0.1:42155/ using document root <none> and password file <none>
I20260620 23:08:39.411813 1153 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 23:08:39.411957 1153 webserver.cc:460] Webserver started at http://127.0.0.1:38363/ using document root <none> and password file <none>
2026-06-20T23:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:39Z Disabled control of system clock
I20260620 23:08:39.420063 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46179
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:33543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:46179
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42155/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 23:08:39.496222 6459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:39.496392 6459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:39.496409 6459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:39.497514 6459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:39.497547 6459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:39.497560 6459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:39.497573 6459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:39.497592 6459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:08:39.497604 6459 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:08:39.497617 6459 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42155/jwks.json
I20260620 23:08:39.498860 6459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:46179
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42155/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:39.499046 6459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:39.499248 6459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:39.503780 6467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:39.503793 6464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:39.503793 6465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:39.504065 6459 server_base.cc:1034] running on GCE node
I20260620 23:08:39.504320 6459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:39.504602 6459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:39.505772 6459 hybrid_clock.cc:648] HybridClock initialized: now 1781996919505756 us; error 44 us; skew 500 ppm
I20260620 23:08:39.506986 6459 webserver.cc:460] Webserver started at http://127.1.32.126:37399/ using document root <none> and password file <none>
I20260620 23:08:39.507206 6459 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:39.507270 6459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:39.507397 6459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:39.508378 6459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "a8924ed8780544ffb23907b9b0bf1377"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "17dcc640d00f344624db6c9209333663"
server_key_iv: "1ac5cd94994407d9253f37225a9e6fc6"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.508737 6459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "a8924ed8780544ffb23907b9b0bf1377"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "17dcc640d00f344624db6c9209333663"
server_key_iv: "1ac5cd94994407d9253f37225a9e6fc6"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.509984 6459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:39.510591 6473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:39.510756 6459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:39.510847 6459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "a8924ed8780544ffb23907b9b0bf1377"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "17dcc640d00f344624db6c9209333663"
server_key_iv: "1ac5cd94994407d9253f37225a9e6fc6"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.510951 6459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:39.519239 6459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:39.519501 6459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:39.519649 6459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:39.523516 6459 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:46179
I20260620 23:08:39.523550 6525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:46179 every 8 connection(s)
I20260620 23:08:39.523881 6459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:39.524514 6526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:39.527060 6526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377: Bootstrap starting.
I20260620 23:08:39.527559 6526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:39.527786 6526 log.cc:824] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:39.528401 6526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377: No bootstrap required, opened a new log
I20260620 23:08:39.529888 6526 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8924ed8780544ffb23907b9b0bf1377" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46179 } }
I20260620 23:08:39.530027 6526 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:39.530049 6526 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8924ed8780544ffb23907b9b0bf1377, State: Initialized, Role: FOLLOWER
I20260620 23:08:39.530145 6526 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8924ed8780544ffb23907b9b0bf1377" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46179 } }
I20260620 23:08:39.530195 6526 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:39.530215 6526 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:39.530241 6526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:39.530711 6526 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8924ed8780544ffb23907b9b0bf1377" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46179 } }
I20260620 23:08:39.530792 6526 leader_election.cc:302] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8924ed8780544ffb23907b9b0bf1377; no voters:
I20260620 23:08:39.530985 6526 leader_election.cc:288] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:39.531025 6529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:39.531201 6529 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [term 1 LEADER]: Becoming Leader. State: Replica: a8924ed8780544ffb23907b9b0bf1377, State: Running, Role: LEADER
I20260620 23:08:39.531277 6526 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:39.531324 6529 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8924ed8780544ffb23907b9b0bf1377" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46179 } }
I20260620 23:08:39.531879 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8924ed8780544ffb23907b9b0bf1377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8924ed8780544ffb23907b9b0bf1377" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46179 } } }
I20260620 23:08:39.531977 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:39.532135 6531 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8924ed8780544ffb23907b9b0bf1377. Latest consensus state: current_term: 1 leader_uuid: "a8924ed8780544ffb23907b9b0bf1377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8924ed8780544ffb23907b9b0bf1377" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46179 } } }
I20260620 23:08:39.532191 6531 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:39.532341 6538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:39.532791 6538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:39.533970 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6459
I20260620 23:08:39.534089 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:39.534258 1153 external_mini_cluster.cc:1428] Setting key 3df6ec6afa251e6c0ef146b823191c49
I20260620 23:08:39.535310 6538 catalog_manager.cc:1347] Generated new cluster ID: c487fa76180947e5813782e2f5467097
I20260620 23:08:39.535362 6538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:39.540395 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.534719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.1.32.126:46179)
0620 23:08:39.534939 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.534948 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.534982 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:08:39.535105 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.535112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.535182 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.535329 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:39.535649 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.537012 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.538173 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.538770 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.538873 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.539749 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:39.539753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:39.539753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:39.539933 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:39.539975 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:39.539988 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:39.540049 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:39.540143 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 23:08:39.540214 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260620 23:08:39.541216 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:39.551728 6538 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:39.552369 6538 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:39.557752 6538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8924ed8780544ffb23907b9b0bf1377: Generated new TSK 0
I20260620 23:08:39.557912 6538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:39.566438 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.562823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.1.32.126:46179)
0620 23:08:39.562880 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.562898 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.563014 (+ 116us) server_negotiation.cc:407] Connection header received
0620 23:08:39.563049 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.563050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.563062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.563139 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:08:39.563262 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.564033 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.564787 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.565103 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.565172 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.565313 (+ 141us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:08:39.566236 (+ 923us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 23:08:39.566364 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260620 23:08:39.566488 6549 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 23:08:39.566581 6555 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:39.562747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46179 (local address 127.0.0.1:45998)
0620 23:08:39.562928 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:39.562939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:39.563005 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:39.563143 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:39.563144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:39.563145 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:08:39.563193 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.563195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.564107 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.564108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.564698 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.564699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.565177 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.565178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.565216 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.565220 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:08:39.566348 (+ 1128us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 23:08:39.566441 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.32.126:46179: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":99,"threads_started":1}
I20260620 23:08:39.568113 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6459
E20260620 23:08:39.577606 6556 webserver.cc:533] Webserver: sslize failed:
E20260620 23:08:39.577694 6556 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T23:08:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (194 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T23:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:39Z Disabled control of system clock
I20260620 23:08:39.612545 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:44709
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:35747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:44709
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/testchainca.cert with env {}
W20260620 23:08:39.687846 6560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:39.688067 6560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:39.688109 6560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:39.689215 6560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:39.689276 6560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:39.689299 6560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:39.689329 6560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:39.689357 6560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 23:08:39.689380 6560 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 23:08:39.689409 6560 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 23:08:39.690838 6560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:35747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:44709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:39.691120 6560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:39.691326 6560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:39.695732 6566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:39.695729 6565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:39.695755 6568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:39.695904 6560 server_base.cc:1034] running on GCE node
I20260620 23:08:39.696158 6560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:39.696432 6560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:39.697599 6560 hybrid_clock.cc:648] HybridClock initialized: now 1781996919697589 us; error 46 us; skew 500 ppm
I20260620 23:08:39.698725 6560 webserver.cc:460] Webserver started at http://127.1.32.126:42271/ using document root <none> and password file <none>
I20260620 23:08:39.699010 6560 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:39.699076 6560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:39.699219 6560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:39.700183 6560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "aa145df3f3504c5d90f00079047f0efa"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "b5d0950aa62374a3bfb141718a828216"
server_key_iv: "2477e9c1f9ae259d8f746974acac1fc1"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.700519 6560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "aa145df3f3504c5d90f00079047f0efa"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "b5d0950aa62374a3bfb141718a828216"
server_key_iv: "2477e9c1f9ae259d8f746974acac1fc1"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.701813 6560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:39.702523 6574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:39.702680 6560 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:39.702775 6560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "aa145df3f3504c5d90f00079047f0efa"
format_stamp: "Formatted at 2026-06-20 23:08:39 on dist-test-slave-fzjm"
server_key: "b5d0950aa62374a3bfb141718a828216"
server_key_iv: "2477e9c1f9ae259d8f746974acac1fc1"
server_key_version: "encryptionkey@0"
I20260620 23:08:39.702857 6560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:39.721884 6560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:39.722172 6560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:39.722313 6560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:39.726352 6560 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:44709
I20260620 23:08:39.726392 6626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:44709 every 8 connection(s)
I20260620 23:08:39.726725 6560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:39.727707 6627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:39.729887 6627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa: Bootstrap starting.
I20260620 23:08:39.730484 6627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:39.730760 6627 log.cc:824] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:39.731478 6627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa: No bootstrap required, opened a new log
I20260620 23:08:39.732949 6627 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa145df3f3504c5d90f00079047f0efa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44709 } }
I20260620 23:08:39.733083 6627 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:39.733105 6627 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa145df3f3504c5d90f00079047f0efa, State: Initialized, Role: FOLLOWER
I20260620 23:08:39.733235 6627 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa145df3f3504c5d90f00079047f0efa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44709 } }
I20260620 23:08:39.733297 6627 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:39.733315 6627 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:39.733369 6627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:39.733855 6627 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa145df3f3504c5d90f00079047f0efa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44709 } }
I20260620 23:08:39.733953 6627 leader_election.cc:302] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa145df3f3504c5d90f00079047f0efa; no voters:
I20260620 23:08:39.734122 6627 leader_election.cc:288] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:39.734185 6630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:39.734335 6630 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [term 1 LEADER]: Becoming Leader. State: Replica: aa145df3f3504c5d90f00079047f0efa, State: Running, Role: LEADER
I20260620 23:08:39.734350 6627 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:39.734452 6630 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa145df3f3504c5d90f00079047f0efa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44709 } }
I20260620 23:08:39.734794 6632 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa145df3f3504c5d90f00079047f0efa. Latest consensus state: current_term: 1 leader_uuid: "aa145df3f3504c5d90f00079047f0efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa145df3f3504c5d90f00079047f0efa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44709 } } }
I20260620 23:08:39.734871 6631 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa145df3f3504c5d90f00079047f0efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa145df3f3504c5d90f00079047f0efa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44709 } } }
I20260620 23:08:39.734906 6632 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:39.734947 6631 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:39.735206 6639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:39.735718 6639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:39.736351 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6560
I20260620 23:08:39.736460 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:39.736596 1153 external_mini_cluster.cc:1428] Setting key 9ffabf208c095e89959b6b5ba0a8a83c
I20260620 23:08:39.737298 6639 catalog_manager.cc:1347] Generated new cluster ID: 197b21214c0849f29b7e69489a1e7ce3
I20260620 23:08:39.737371 6639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:39.744436 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.737123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.1.32.126:44709)
0620 23:08:39.739051 (+ 1928us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.739059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.739101 (+ 42us) server_negotiation.cc:407] Connection header received
0620 23:08:39.739210 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.739217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.739283 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.739421 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:39.739650 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.741047 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.742038 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.742840 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.743003 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.743742 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:39.743746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:39.743746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:08:39.743943 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:39.743993 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:08:39.744008 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:39.744077 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:39.744149 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 23:08:39.744236 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1819,"thread_start_us":67,"threads_started":1}
I20260620 23:08:39.745510 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:08:39.745802 6639 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:39.746474 6639 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:39.750787 6639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa145df3f3504c5d90f00079047f0efa: Generated new TSK 0
I20260620 23:08:39.750958 6639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:39.792642 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:39.760516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.1.32.126:44709)
0620 23:08:39.760556 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:39.760557 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:39.760648 (+ 91us) server_negotiation.cc:407] Connection header received
0620 23:08:39.760705 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:39.760705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:39.760713 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:39.760763 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0620 23:08:39.760886 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.761327 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.762338 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.762491 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.762535 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.762692 (+ 157us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 23:08:39.792396 (+ 29704us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 23:08:39.792583 (+ 187us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33138: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 23:08:39.792734 6650 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33138: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 23:08:39.792807 6656 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:39.760414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:44709 (local address 127.0.0.1:33138)
0620 23:08:39.760573 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:39.760584 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:39.760643 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:39.760770 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:39.760771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:39.760773 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 23:08:39.760832 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.760834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.761409 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.761410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.762263 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:39.762265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:39.762553 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:39.762554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:39.762612 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:39.762617 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 23:08:39.792572 (+ 29955us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 23:08:39.792705 (+ 133us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.32.126:44709: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260620 23:08:39.793135 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6560
2026-06-20T23:08:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (204 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:40 dist-test-slave-fzjm krb5kdc[6668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:40 dist-test-slave-fzjm krb5kdc[6668](info): set up 2 sockets
Jun 20 23:08:40 dist-test-slave-fzjm krb5kdc[6668](info): commencing operation
krb5kdc: starting...
W20260620 23:08:40.962569 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:41.067924 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36475
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:36475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:41.144305 6681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:41.144611 6681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:41.144680 6681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:41.145817 6681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:41.145885 6681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:41.145912 6681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:41.145943 6681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:41.145967 6681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:41.147568 6681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:36475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:41.147848 6681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:41.148095 6681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:41.152738 6687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:41.152825 6681 server_base.cc:1034] running on GCE node
W20260620 23:08:41.152740 6689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:41.152863 6686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:41.153174 6681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:41.153420 6681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:41.154578 6681 hybrid_clock.cc:648] HybridClock initialized: now 1781996921154564 us; error 30 us; skew 500 ppm
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.160741 6681 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:41.161175 6681 webserver.cc:460] Webserver started at http://127.1.32.126:43603/ using document root <none> and password file <none>
I20260620 23:08:41.161418 6681 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:41.161487 6681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:41.161620 6681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:41.162521 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "b7b1e1585d074499ac0d83f7d2de2b9c"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "e64f840e9f2d43bea779992869555a09"
server_key_iv: "f4210de3d3e6ef4cce38d932563083fa"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.162843 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "b7b1e1585d074499ac0d83f7d2de2b9c"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "e64f840e9f2d43bea779992869555a09"
server_key_iv: "f4210de3d3e6ef4cce38d932563083fa"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.164077 6681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:41.164814 6696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.164983 6681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:41.165074 6681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "b7b1e1585d074499ac0d83f7d2de2b9c"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "e64f840e9f2d43bea779992869555a09"
server_key_iv: "f4210de3d3e6ef4cce38d932563083fa"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.165149 6681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:41.181903 6681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:41.183640 6681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:41.183784 6681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:41.187675 6681 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:36475
I20260620 23:08:41.187703 6748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:36475 every 8 connection(s)
I20260620 23:08:41.188004 6681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:41.188807 6749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:41.190945 6749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c: Bootstrap starting.
I20260620 23:08:41.191474 6749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:41.191793 6749 log.cc:824] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:41.191828 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6681
I20260620 23:08:41.191915 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:41.192090 1153 external_mini_cluster.cc:1428] Setting key cc65ae24b50769948d53b302437f7023
I20260620 23:08:41.192567 6749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c: No bootstrap required, opened a new log
I20260620 23:08:41.194144 6749 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36475 } }
I20260620 23:08:41.194290 6749 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:41.194339 6749 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b1e1585d074499ac0d83f7d2de2b9c, State: Initialized, Role: FOLLOWER
I20260620 23:08:41.194468 6749 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36475 } }
I20260620 23:08:41.194556 6749 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:41.194588 6749 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:41.194635 6749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:41.195657 6749 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36475 } }
I20260620 23:08:41.195737 6749 leader_election.cc:302] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7b1e1585d074499ac0d83f7d2de2b9c; no voters:
I20260620 23:08:41.195889 6749 leader_election.cc:288] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:41.195935 6754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:41.196059 6754 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [term 1 LEADER]: Becoming Leader. State: Replica: b7b1e1585d074499ac0d83f7d2de2b9c, State: Running, Role: LEADER
I20260620 23:08:41.196141 6749 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:41.196142 6754 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36475 } }
I20260620 23:08:41.196448 6755 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36475 } } }
I20260620 23:08:41.196554 6755 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:41.196444 6756 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7b1e1585d074499ac0d83f7d2de2b9c. Latest consensus state: current_term: 1 leader_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b1e1585d074499ac0d83f7d2de2b9c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36475 } } }
I20260620 23:08:41.196677 6756 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:41.196729 6761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:41.197161 6761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:41.199092 6761 catalog_manager.cc:1347] Generated new cluster ID: 369b028d86744e8b9b725f0a0a09c0f3
I20260620 23:08:41.199146 6761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:41.204407 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.192555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36026 (local address 127.1.32.126:36475)
0620 23:08:41.192718 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.192726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.192756 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:08:41.194135 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.194142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.194191 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.194304 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.195218 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.196288 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.197265 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.197815 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.198189 (+ 374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.200736 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:41.200739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:41.200739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:41.200750 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:41.202639 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.203102 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.203105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.203105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.203211 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.203433 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.203434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.203434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.203691 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:41.203824 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:41.203929 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 23:08:41.204184 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260620 23:08:41.221575 6761 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:41.222100 6761 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:41.227044 6761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7b1e1585d074499ac0d83f7d2de2b9c: Generated new TSK 0
I20260620 23:08:41.227221 6761 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:41.253109 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:36475
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:41.333062 6775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:41.333251 6775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:41.333273 6775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:41.334364 6775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:41.334404 6775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:41.334453 6775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:41.335878 6775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:41.336187 6775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:41.336433 6775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:41.341486 6783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:41.341539 6781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:41.341486 6780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:41.341663 6775 server_base.cc:1034] running on GCE node
I20260620 23:08:41.341871 6775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:41.342103 6775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:41.343235 6775 hybrid_clock.cc:648] HybridClock initialized: now 1781996921343225 us; error 26 us; skew 500 ppm
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.345593 6775 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:41.345984 6775 webserver.cc:460] Webserver started at http://127.1.32.65:38475/ using document root <none> and password file <none>
I20260620 23:08:41.346208 6775 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:41.346271 6775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:41.346426 6775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:41.347532 6775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "59b2d19eebe94ceebe39821525103940"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "890b960a4226f8be92b4490cb624c728"
server_key_iv: "e6eff836da1ea267c28bf554cd50e5c3"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.347864 6775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "59b2d19eebe94ceebe39821525103940"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "890b960a4226f8be92b4490cb624c728"
server_key_iv: "e6eff836da1ea267c28bf554cd50e5c3"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.349112 6775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:41.349785 6790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.349946 6775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:41.350050 6775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "59b2d19eebe94ceebe39821525103940"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "890b960a4226f8be92b4490cb624c728"
server_key_iv: "e6eff836da1ea267c28bf554cd50e5c3"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.350119 6775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:41.357641 6775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:41.359548 6775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:41.359690 6775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:41.359908 6775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:41.360185 6775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:41.360236 6775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.360272 6775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:41.360298 6775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.365646 6775 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:40447
I20260620 23:08:41.365736 6903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:40447 every 8 connection(s)
I20260620 23:08:41.365995 6775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:41.367740 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6775
I20260620 23:08:41.367839 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:41.368005 1153 external_mini_cluster.cc:1428] Setting key a321bc20680cd294b89e63269c0eed02
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:41.376139 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.366629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:36475 (local address 127.1.32.65:48245)
0620 23:08:41.366766 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:41.366803 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:41.367828 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:41.368085 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:41.368089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:41.368415 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:41.368660 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:41.368666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.369348 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.369349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:41.370598 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:41.370601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.371048 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.371049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:41.371185 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.371630 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:41.371639 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:41.373061 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:41.374923 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:41.374935 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:41.374936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:41.375139 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:41.375354 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:41.375355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:41.375355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:41.375441 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:41.375708 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:41.375712 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:41.375813 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 23:08:41.375888 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:08:41.375995 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260620 23:08:41.376184 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.366660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:48245 (local address 127.1.32.126:36475)
0620 23:08:41.366714 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.366716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.366881 (+ 165us) server_negotiation.cc:407] Connection header received
0620 23:08:41.367931 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.367932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.367954 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.368014 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.368732 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.369260 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.370703 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.370966 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.371033 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.373150 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:41.373152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:41.373153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:41.373165 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:41.374806 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.375212 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.375213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.375214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.375285 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.375508 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.375509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.375509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.375639 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:41.375715 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:41.375938 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 23:08:41.376040 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:08:41.376668 6904 heartbeater.cc:347] Connected to a master server at 127.1.32.126:36475
I20260620 23:08:41.376760 6904 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:41.376936 6904 heartbeater.cc:511] Master 127.1.32.126:36475 requested a full tablet report, sending...
I20260620 23:08:41.377439 6713 ts_manager.cc:194] Registered new tserver with Master: 59b2d19eebe94ceebe39821525103940 (127.1.32.65:40447)
I20260620 23:08:41.378110 6713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:48245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:41.391495 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:36475
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:41.469643 6909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:41.469882 6909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:41.469924 6909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:41.471086 6909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:41.471151 6909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:41.471200 6909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:41.472694 6909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:41.472960 6909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:41.473183 6909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:41.478089 6914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:41.478068 6917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:41.478196 6909 server_base.cc:1034] running on GCE node
W20260620 23:08:41.478039 6915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:41.478521 6909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:41.478736 6909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:41.479908 6909 hybrid_clock.cc:648] HybridClock initialized: now 1781996921479898 us; error 26 us; skew 500 ppm
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.482204 6909 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:41.482625 6909 webserver.cc:460] Webserver started at http://127.1.32.66:45329/ using document root <none> and password file <none>
I20260620 23:08:41.482879 6909 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:41.482996 6909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:41.483151 6909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:41.484237 6909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "62639e447bce418095ce538d03316ad5"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "5d6baceaa3cf19d63410b9e0cdc768ce"
server_key_iv: "25e949aa3096a1070bb0169c92a3cbed"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.484606 6909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "62639e447bce418095ce538d03316ad5"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "5d6baceaa3cf19d63410b9e0cdc768ce"
server_key_iv: "25e949aa3096a1070bb0169c92a3cbed"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.485966 6909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:41.486701 6924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.486860 6909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:41.486984 6909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "62639e447bce418095ce538d03316ad5"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "5d6baceaa3cf19d63410b9e0cdc768ce"
server_key_iv: "25e949aa3096a1070bb0169c92a3cbed"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.487058 6909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:41.498816 6909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:41.500667 6909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:41.500836 6909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:41.501075 6909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:41.501423 6909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:41.501475 6909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.501511 6909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:41.501534 6909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.507054 6909 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:46429
I20260620 23:08:41.507192 7037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:46429 every 8 connection(s)
I20260620 23:08:41.507403 6909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:41.515856 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 6909
I20260620 23:08:41.515954 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:41.516160 1153 external_mini_cluster.cc:1428] Setting key 774186c089e533fc1e3a93cae7ed42e4
I20260620 23:08:41.516923 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.508134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:55223 (local address 127.1.32.126:36475)
0620 23:08:41.508199 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.508201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.508476 (+ 275us) server_negotiation.cc:407] Connection header received
0620 23:08:41.509445 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.509447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.509459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.509522 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.510164 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.510730 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.511811 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.512028 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.512118 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.514224 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:41.514225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:41.514225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:41.514232 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:41.515468 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.515838 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.515840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.515840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.515900 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.516165 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.516166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.516166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.516299 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:41.516377 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:41.516711 (+ 334us) server_negotiation.cc:299] Negotiation successful
0620 23:08:41.516813 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:08:41.517112 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.508099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:36475 (local address 127.1.32.66:55223)
0620 23:08:41.508344 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:41.508388 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:41.509331 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:41.509607 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:41.509610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:41.509865 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:41.510056 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:41.510061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.510825 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.510826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:41.511710 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:41.511712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.512138 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.512139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:41.512272 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.512735 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:41.512743 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:41.514142 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:41.515540 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:41.515545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:41.515546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:41.515763 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:41.515983 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:41.515984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:41.515984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:41.516098 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:41.516380 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:41.516387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:41.516562 (+ 175us) client_negotiation.cc:769] Sending connection context
0620 23:08:41.516675 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 23:08:41.516861 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":108,"threads_started":1}
I20260620 23:08:41.517561 7038 heartbeater.cc:347] Connected to a master server at 127.1.32.126:36475
I20260620 23:08:41.517666 7038 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:41.517845 7038 heartbeater.cc:511] Master 127.1.32.126:36475 requested a full tablet report, sending...
I20260620 23:08:41.518263 6713 ts_manager.cc:194] Registered new tserver with Master: 62639e447bce418095ce538d03316ad5 (127.1.32.66:46429)
I20260620 23:08:41.518646 6713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:55223
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:41.540889 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:36475
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:41.617928 7043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:41.618124 7043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:41.618146 7043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:41.619256 7043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:41.619294 7043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:41.619319 7043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:41.620622 7043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:41.620847 7043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:41.621104 7043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:41.626415 7049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:41.626415 7048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:41.626634 7043 server_base.cc:1034] running on GCE node
W20260620 23:08:41.626595 7051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:41.627002 7043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:41.627276 7043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:41.628428 7043 hybrid_clock.cc:648] HybridClock initialized: now 1781996921628417 us; error 27 us; skew 500 ppm
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.630754 7043 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:41.631210 7043 webserver.cc:460] Webserver started at http://127.1.32.67:45571/ using document root <none> and password file <none>
I20260620 23:08:41.631384 7043 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:41.631423 7043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:41.631507 7043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:41.632527 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "39d05a6db16e4e3ea6505d0c6429192c"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "ee38c5d5e1e74bc71239f45d0ed9c50c"
server_key_iv: "b1ce119965023b7e6d352219bad43974"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.632812 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "39d05a6db16e4e3ea6505d0c6429192c"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "ee38c5d5e1e74bc71239f45d0ed9c50c"
server_key_iv: "b1ce119965023b7e6d352219bad43974"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.634012 7043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:41.634665 7058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.634822 7043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:41.634940 7043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "39d05a6db16e4e3ea6505d0c6429192c"
format_stamp: "Formatted at 2026-06-20 23:08:41 on dist-test-slave-fzjm"
server_key: "ee38c5d5e1e74bc71239f45d0ed9c50c"
server_key_iv: "b1ce119965023b7e6d352219bad43974"
server_key_version: "encryptionkey@0"
I20260620 23:08:41.635004 7043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:41.658280 7043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:41.660491 7043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:41.660601 7043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:41.660789 7043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:41.661124 7043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:41.661166 7043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.661190 7043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:41.661237 7043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:41.666656 7043 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39741
I20260620 23:08:41.666754 7171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39741 every 8 connection(s)
I20260620 23:08:41.667070 7043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:41.675483 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 7043
I20260620 23:08:41.675608 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:41.675793 1153 external_mini_cluster.cc:1428] Setting key c412efffcbcd61ed3813de7724f3ef26
I20260620 23:08:41.676626 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.667932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:57979 (local address 127.1.32.126:36475)
0620 23:08:41.667996 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.667998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.668221 (+ 223us) server_negotiation.cc:407] Connection header received
0620 23:08:41.669223 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.669224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.669234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.669295 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.669840 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.670324 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.671370 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.671555 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.671615 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.673522 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:41.673523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:41.673524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:41.673529 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:41.675125 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.675595 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.675596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.675597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.675656 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.675941 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.675942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.675942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.676063 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:41.676140 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:41.676466 (+ 326us) server_negotiation.cc:299] Negotiation successful
0620 23:08:41.676537 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:08:41.676869 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.667911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:36475 (local address 127.1.32.67:57979)
0620 23:08:41.668094 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:41.668135 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:41.669126 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:41.669347 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:41.669350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:41.669584 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:41.669771 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:41.669775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.670403 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.670404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:41.671297 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:41.671299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.671628 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.671628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:41.671718 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.672135 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:41.672143 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:41.673445 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:41.675197 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:41.675202 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:41.675203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:41.675514 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:41.675736 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:41.675737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:41.675737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:41.675841 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:41.676151 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:41.676158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:41.676317 (+ 159us) client_negotiation.cc:769] Sending connection context
0620 23:08:41.676436 (+ 119us) client_negotiation.cc:240] Negotiation successful
0620 23:08:41.676586 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260620 23:08:41.677327 7172 heartbeater.cc:347] Connected to a master server at 127.1.32.126:36475
I20260620 23:08:41.677409 7172 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:41.677546 7172 heartbeater.cc:511] Master 127.1.32.126:36475 requested a full tablet report, sending...
I20260620 23:08:41.677881 6713 ts_manager.cc:194] Registered new tserver with Master: 39d05a6db16e4e3ea6505d0c6429192c (127.1.32.67:39741)
I20260620 23:08:41.678233 6713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:57979
I20260620 23:08:41.687659 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/test-admin.keytab.
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.701436 1153 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 23:08:41.703087 7179 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:41.702297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.1.32.65:40447)
0620 23:08:41.702470 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.702476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.702486 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:41.702518 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.702519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.702534 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.702614 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.702992 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44750: BlockingRecv error: recv got EOF from 127.0.0.1:44750 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
W20260620 23:08:41.703218 1153 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc' of type FILE
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.704262 1153 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.705116 1153 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:41.712136 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.705584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.1.32.65:40447)
0620 23:08:41.705650 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.705651 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.705718 (+ 67us) server_negotiation.cc:407] Connection header received
0620 23:08:41.705758 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.705758 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.705767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.705812 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.706174 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.706836 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.707533 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.707823 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.707877 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.709864 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:41.709870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:41.709871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:41.709876 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:41.711235 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.711531 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.711533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.711534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.711583 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.711728 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.711729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.711729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.711834 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:41.711922 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:41.711991 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:08:41.712065 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 23:08:41.713845 7179 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:41.713347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.1.32.65:40447)
0620 23:08:41.713398 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.713399 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.713434 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:08:41.713476 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.713494 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.713502 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.713549 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.713798 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44758: BlockingRecv error: recv got EOF from 127.0.0.1:44758 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260620 23:08:41.713925 1153 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc' of type FILE
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.714736 1153 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.715612 1153 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:41.722119 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.716077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.1.32.65:40447)
0620 23:08:41.716137 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.716139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.716216 (+ 77us) server_negotiation.cc:407] Connection header received
0620 23:08:41.716298 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.716299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.716310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.716364 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.716706 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.717174 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.717849 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.718049 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.718131 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.719940 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:41.719941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:41.719942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:41.719948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:41.721045 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.721388 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.721389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.721389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.721454 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.721673 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.721674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.721674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.721765 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:41.721858 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:41.721960 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:08:41.722030 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 23:08:41.723598 7179 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:41.723056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.1.32.65:40447)
0620 23:08:41.723093 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.723094 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.723154 (+ 60us) server_negotiation.cc:407] Connection header received
0620 23:08:41.723221 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.723221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.723229 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.723280 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.723562 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44762: BlockingRecv error: recv got EOF from 127.0.0.1:44762 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:08:41.723766 1153 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996897900784-1153-0/krb5kdc/krb5cc)
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.724620 1153 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:41.725459 1153 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[6668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:08:41.731720 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:41.725914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44770 (local address 127.1.32.65:40447)
0620 23:08:41.725995 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:41.725998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:41.726018 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:08:41.726073 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:41.726073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:41.726080 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:41.726124 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:41.726437 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.726863 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.727653 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:41.727818 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:41.727868 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:41.729696 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:41.729697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:41.729697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:41.729702 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:41.730759 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.731094 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.731094 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.731095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.731145 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:41.731329 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:41.731329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:41.731330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:41.731400 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:41.731473 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:41.731542 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:08:41.731592 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:08:41.732216 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6775
I20260620 23:08:41.737396 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6909
I20260620 23:08:41.741725 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 7043
I20260620 23:08:41.745765 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 6681
2026-06-20T23:08:41Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1783 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[7192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[7192](info): set up 2 sockets
Jun 20 23:08:41 dist-test-slave-fzjm krb5kdc[7192](info): commencing operation
krb5kdc: starting...
W20260620 23:08:42.743058 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:42 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab.
Entry for principal oryx/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab.
I20260620 23:08:42.849654 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:35483
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab
--principal=oryx/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:35483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:42.924778 7205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:42.924958 7205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:42.924978 7205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:42.926028 7205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:42.926061 7205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:42.926075 7205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:42.926087 7205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:42.926100 7205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:42.927482 7205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:35483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab
--principal=oryx/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:42.927659 7205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:42.927812 7205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:42.932283 7211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:42.932252 7213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:42.932286 7205 server_base.cc:1034] running on GCE node
W20260620 23:08:42.932260 7210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:42.932773 7205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:42.933059 7205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:42.934203 7205 hybrid_clock.cc:648] HybridClock initialized: now 1781996922934190 us; error 27 us; skew 500 ppm
Jun 20 23:08:42 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:42.936663 7205 init.cc:377] Logged in from keytab as oryx/127.1.32.126@KRBTEST.COM (short username oryx)
I20260620 23:08:42.937124 7205 webserver.cc:460] Webserver started at http://127.1.32.126:38239/ using document root <none> and password file <none>
I20260620 23:08:42.937355 7205 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:42.937400 7205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:42.937577 7205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:42.938584 7205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "6733ba54101e4f608761e9288c8d5f4c"
format_stamp: "Formatted at 2026-06-20 23:08:42 on dist-test-slave-fzjm"
server_key: "7787cbf8097b26c47e3430ecebeb983c"
server_key_iv: "668adc4071e14af8ebd2e75c6da6c1db"
server_key_version: "encryptionkey@0"
I20260620 23:08:42.938983 7205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "6733ba54101e4f608761e9288c8d5f4c"
format_stamp: "Formatted at 2026-06-20 23:08:42 on dist-test-slave-fzjm"
server_key: "7787cbf8097b26c47e3430ecebeb983c"
server_key_iv: "668adc4071e14af8ebd2e75c6da6c1db"
server_key_version: "encryptionkey@0"
I20260620 23:08:42.940279 7205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:42.940961 7220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:42.941138 7205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:42.941231 7205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "6733ba54101e4f608761e9288c8d5f4c"
format_stamp: "Formatted at 2026-06-20 23:08:42 on dist-test-slave-fzjm"
server_key: "7787cbf8097b26c47e3430ecebeb983c"
server_key_iv: "668adc4071e14af8ebd2e75c6da6c1db"
server_key_version: "encryptionkey@0"
I20260620 23:08:42.941314 7205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:42.963376 7205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:42.965351 7205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:42.965503 7205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:42.969426 7205 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:35483
I20260620 23:08:42.969460 7272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:35483 every 8 connection(s)
I20260620 23:08:42.969807 7205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:42.970709 7273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:42.972786 7273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: Bootstrap starting.
I20260620 23:08:42.973371 7273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:42.973536 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 7205
I20260620 23:08:42.973640 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:42.973706 7273 log.cc:824] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:42.973841 1153 external_mini_cluster.cc:1428] Setting key 5dade1d223510cee541e1ac6c1c1b216
I20260620 23:08:42.974551 7273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: No bootstrap required, opened a new log
I20260620 23:08:42.977144 7273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } }
I20260620 23:08:42.977294 7273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:42.977326 7273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6733ba54101e4f608761e9288c8d5f4c, State: Initialized, Role: FOLLOWER
I20260620 23:08:42.977463 7273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } }
I20260620 23:08:42.977560 7273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:42.977589 7273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:42.977635 7273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:42.978245 7273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } }
I20260620 23:08:42.978353 7273 leader_election.cc:302] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6733ba54101e4f608761e9288c8d5f4c; no voters:
I20260620 23:08:42.978515 7273 leader_election.cc:288] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:42.978562 7278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:42.978699 7278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 1 LEADER]: Becoming Leader. State: Replica: 6733ba54101e4f608761e9288c8d5f4c, State: Running, Role: LEADER
I20260620 23:08:42.978773 7273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:42.978782 7278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } }
I20260620 23:08:42.979163 7279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } }
I20260620 23:08:42.979352 7279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:42.979171 7280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6733ba54101e4f608761e9288c8d5f4c. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } }
I20260620 23:08:42.979959 7280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
W20260620 23:08:42.980707 7294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:42.980768 7294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:42.980821 7287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:42.980973 7287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:08:42 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:42.982946 7287 catalog_manager.cc:1347] Generated new cluster ID: 3d778f7abbef4e43944afd7af343d278
I20260620 23:08:42.982995 7287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:42.984954 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:42.974336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.1.32.126:35483)
0620 23:08:42.974514 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:42.974523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:42.974554 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:08:42.975134 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:42.975142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:42.975204 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:42.975370 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:42.975951 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:42.977321 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:42.978368 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:42.979077 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:42.979495 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:42.981873 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:42.981877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:42.981877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:42.981887 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:42.983508 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:42.983852 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:42.983855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:42.983855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:42.983939 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:42.984144 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:42.984144 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:42.984145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:42.984371 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:42.984520 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:42.984632 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 23:08:42.984763 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260620 23:08:43.015077 7287 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:43.015615 7287 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:43.029639 7287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: Generated new TSK 0
I20260620 23:08:43.029870 7287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab.
Entry for principal oryx/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab.
I20260620 23:08:43.057909 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:35483
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab
--principal=oryx/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:43.136951 7299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:43.137163 7299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:43.137200 7299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:43.138305 7299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:43.138365 7299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:43.138408 7299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:43.139798 7299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab
--principal=oryx/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:43.140028 7299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:43.140230 7299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:43.145138 7304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.145160 7307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.145126 7305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.145414 7299 server_base.cc:1034] running on GCE node
I20260620 23:08:43.145635 7299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:43.145869 7299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:43.147032 7299 hybrid_clock.cc:648] HybridClock initialized: now 1781996923147029 us; error 39 us; skew 500 ppm
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:43.149474 7299 init.cc:377] Logged in from keytab as oryx/127.1.32.65@KRBTEST.COM (short username oryx)
I20260620 23:08:43.149871 7299 webserver.cc:460] Webserver started at http://127.1.32.65:35633/ using document root <none> and password file <none>
I20260620 23:08:43.150106 7299 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.150151 7299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.150291 7299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:43.151346 7299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "83163b27b5d94c869492f1391b362a24"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "cb4fcbc9953fd62f1954dc8931218ec5"
server_key_iv: "d4d144ec830d90b5bdfd6429107082f3"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.151687 7299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "83163b27b5d94c869492f1391b362a24"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "cb4fcbc9953fd62f1954dc8931218ec5"
server_key_iv: "d4d144ec830d90b5bdfd6429107082f3"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.153005 7299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:43.153739 7314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.153908 7299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:43.154021 7299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "83163b27b5d94c869492f1391b362a24"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "cb4fcbc9953fd62f1954dc8931218ec5"
server_key_iv: "d4d144ec830d90b5bdfd6429107082f3"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.154114 7299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.168000 7299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:43.169696 7299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:43.169836 7299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:43.170042 7299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:43.170312 7299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:43.170361 7299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.170398 7299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:43.170423 7299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.176604 7299 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:37481
I20260620 23:08:43.176638 7427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:37481 every 8 connection(s)
I20260620 23:08:43.176941 7299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:43.182374 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 7299
I20260620 23:08:43.182471 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:43.182662 1153 external_mini_cluster.cc:1428] Setting key e165e1e3bf15fc05337ef6a31b0ba4ef
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:43.186995 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.177681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:39213 (local address 127.1.32.126:35483)
0620 23:08:43.177735 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.177736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.177955 (+ 219us) server_negotiation.cc:407] Connection header received
0620 23:08:43.178407 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.178408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.178416 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.178462 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.179526 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.180124 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.181285 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.181500 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.181566 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.183833 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.183835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.183835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.183843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.185400 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.185874 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.185875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.185875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.185939 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.186233 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.186234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.186234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.186324 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.186371 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.186748 (+ 377us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.186849 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:08:43.187147 7430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.177639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:39213)
0620 23:08:43.177813 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.177868 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.178327 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.178513 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.178516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.179234 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.179409 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.179414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.180234 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.180235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.181205 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.181207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.181580 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.181581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.181676 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.182201 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.182214 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.183739 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.185487 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.185492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.185494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.185798 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.186021 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.186022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.186022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.186159 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.186386 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.186392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.186546 (+ 154us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.186685 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.186864 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260620 23:08:43.187597 7428 heartbeater.cc:347] Connected to a master server at 127.1.32.126:35483
I20260620 23:08:43.187690 7428 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:43.187857 7428 heartbeater.cc:511] Master 127.1.32.126:35483 requested a full tablet report, sending...
I20260620 23:08:43.188385 7237 ts_manager.cc:194] Registered new tserver with Master: 83163b27b5d94c869492f1391b362a24 (127.1.32.65:37481)
I20260620 23:08:43.188977 7237 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:39213
WARNING: no policy specified for oryx/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab.
Entry for principal oryx/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab.
I20260620 23:08:43.205309 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:35483
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab
--principal=oryx/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:43.283288 7433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:43.283507 7433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:43.283543 7433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:43.284677 7433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:43.284741 7433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:43.284783 7433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:43.286198 7433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab
--principal=oryx/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:43.286449 7433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:43.286666 7433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:43.291368 7438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.291368 7441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.291381 7439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.291757 7433 server_base.cc:1034] running on GCE node
I20260620 23:08:43.291987 7433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:43.292232 7433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:43.293376 7433 hybrid_clock.cc:648] HybridClock initialized: now 1781996923293363 us; error 26 us; skew 500 ppm
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:43.295938 7433 init.cc:377] Logged in from keytab as oryx/127.1.32.66@KRBTEST.COM (short username oryx)
I20260620 23:08:43.296368 7433 webserver.cc:460] Webserver started at http://127.1.32.66:36957/ using document root <none> and password file <none>
I20260620 23:08:43.296649 7433 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.296721 7433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.296880 7433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:43.297930 7433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "8e97115bc9b541b8962c6a974d6f90d6"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "a8076c12408fd13f971637d5ca538bee"
server_key_iv: "9e8ba1884df59d6fb654b9f247e084ab"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.298317 7433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "8e97115bc9b541b8962c6a974d6f90d6"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "a8076c12408fd13f971637d5ca538bee"
server_key_iv: "9e8ba1884df59d6fb654b9f247e084ab"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.299808 7433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:08:43.300539 7448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.300702 7433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:43.300796 7433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "8e97115bc9b541b8962c6a974d6f90d6"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "a8076c12408fd13f971637d5ca538bee"
server_key_iv: "9e8ba1884df59d6fb654b9f247e084ab"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.300884 7433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.311267 7433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:43.313088 7433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:43.313256 7433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:43.313516 7433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:43.313823 7433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:43.313879 7433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.313918 7433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:43.313958 7433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.320287 7433 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:35447
I20260620 23:08:43.320323 7561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:35447 every 8 connection(s)
I20260620 23:08:43.320647 7433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:43.329473 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 7433
I20260620 23:08:43.329602 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:43.329839 1153 external_mini_cluster.cc:1428] Setting key 822d46386aa5fb15bd3c1dffe079a1c4
I20260620 23:08:43.330054 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.321380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:58347 (local address 127.1.32.126:35483)
0620 23:08:43.321463 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.321466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.321680 (+ 214us) server_negotiation.cc:407] Connection header received
0620 23:08:43.322158 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.322159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.322169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.322224 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.323137 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.323684 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.324658 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.324859 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.324915 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.326983 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.326984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.326985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.326992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.328544 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.328876 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.328877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.328877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.328928 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.329208 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.329209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.329209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.329395 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.329485 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.329826 (+ 341us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.329936 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 23:08:43.330202 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.321337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:58347)
0620 23:08:43.321541 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.321598 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.322073 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.322276 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.322279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.322825 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.323058 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.323063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.323781 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.323781 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.324581 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.324583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.324928 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.324929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.325021 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.325479 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.325487 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.326839 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.328613 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.328617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.328619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.328813 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.328993 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.328994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.328994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.329144 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.329476 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.329481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.329674 (+ 193us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.329781 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.329940 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":106,"threads_started":1}
I20260620 23:08:43.330657 7562 heartbeater.cc:347] Connected to a master server at 127.1.32.126:35483
I20260620 23:08:43.330773 7562 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:43.330994 7562 heartbeater.cc:511] Master 127.1.32.126:35483 requested a full tablet report, sending...
I20260620 23:08:43.331408 7237 ts_manager.cc:194] Registered new tserver with Master: 8e97115bc9b541b8962c6a974d6f90d6 (127.1.32.66:35447)
I20260620 23:08:43.331997 7237 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:58347
WARNING: no policy specified for oryx/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab.
Entry for principal oryx/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab.
I20260620 23:08:43.354290 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:35483
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab
--principal=oryx/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:43.432734 7567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:43.432920 7567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:43.432948 7567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:43.433969 7567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:43.434001 7567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:43.434026 7567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:43.435364 7567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab
--principal=oryx/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:43.435532 7567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:43.435699 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:08:43.440373 7567 server_base.cc:1034] running on GCE node
W20260620 23:08:43.440428 7573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.440352 7572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.440358 7575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.440799 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:43.441068 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:43.442196 7567 hybrid_clock.cc:648] HybridClock initialized: now 1781996923442187 us; error 25 us; skew 500 ppm
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:43.444638 7567 init.cc:377] Logged in from keytab as oryx/127.1.32.67@KRBTEST.COM (short username oryx)
I20260620 23:08:43.445029 7567 webserver.cc:460] Webserver started at http://127.1.32.67:45801/ using document root <none> and password file <none>
I20260620 23:08:43.445251 7567 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.445308 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.445477 7567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:43.446440 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "ab2ac687bc4843c095f4a4bc474d289b"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "19b8761241bce0991923742e32caaa00"
server_key_iv: "4857c9f2365662d8543f47ab2b57d8e3"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.446784 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "ab2ac687bc4843c095f4a4bc474d289b"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "19b8761241bce0991923742e32caaa00"
server_key_iv: "4857c9f2365662d8543f47ab2b57d8e3"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.448118 7567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:43.448872 7582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.449100 7567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:43.449183 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "ab2ac687bc4843c095f4a4bc474d289b"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "19b8761241bce0991923742e32caaa00"
server_key_iv: "4857c9f2365662d8543f47ab2b57d8e3"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.449252 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.469213 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:43.471108 7567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:43.471247 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:43.471488 7567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:43.471777 7567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:43.471809 7567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.471829 7567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:43.471874 7567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.477581 7567 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:43369
I20260620 23:08:43.477624 7695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:43369 every 8 connection(s)
I20260620 23:08:43.477926 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:08:43.478544 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 7567
I20260620 23:08:43.478648 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:43.478855 1153 external_mini_cluster.cc:1428] Setting key 33925c386b96cab333095e0418e0802a
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:43.487233 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.478678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:48371 (local address 127.1.32.126:35483)
0620 23:08:43.478724 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.478726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.478976 (+ 250us) server_negotiation.cc:407] Connection header received
0620 23:08:43.479805 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.479806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.479815 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.479851 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.480819 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.481330 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.482321 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.482518 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.482581 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.484577 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.484578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.484578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.484585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.485957 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.486321 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.486322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.486322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.486379 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.486653 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.486654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.486654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.486753 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.486804 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.487106 (+ 302us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.487168 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:08:43.487450 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.478678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:48371)
0620 23:08:43.478826 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.478860 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.479706 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.479912 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.479916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.480515 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.480726 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.480731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.481414 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.481416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.482228 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.482230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.482595 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.482596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.482694 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.483189 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.483197 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.484497 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.486033 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.486039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.486040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.486238 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.486480 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.486481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.486481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.486586 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.486811 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.486815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.486991 (+ 176us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.487081 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.487202 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260620 23:08:43.488031 7696 heartbeater.cc:347] Connected to a master server at 127.1.32.126:35483
I20260620 23:08:43.488112 7696 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:43.488301 7696 heartbeater.cc:511] Master 127.1.32.126:35483 requested a full tablet report, sending...
I20260620 23:08:43.488660 7237 ts_manager.cc:194] Registered new tserver with Master: ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369)
I20260620 23:08:43.489050 7237 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:48371
I20260620 23:08:43.490805 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.32.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.125.keytab.
Entry for principal oryx/127.1.32.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.125.keytab.
I20260620 23:08:43.513310 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:35437
--webserver_interface=127.1.32.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.125.keytab
--principal=oryx/127.1.32.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:35483,127.1.32.125:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:43.592213 7701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:43.592402 7701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:43.592429 7701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:43.593477 7701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:43.593518 7701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:43.593533 7701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:43.593547 7701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:43.593561 7701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:43.594905 7701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:35483,127.1.32.125:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.125.keytab
--principal=oryx/127.1.32.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.32.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
W20260620 23:08:43.595029 7701 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.32.126:35483,127.1.32.125:35437'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 23:08:43.595156 7701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:43.595316 7701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:43.600088 7709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.600072 7707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.600124 7701 server_base.cc:1034] running on GCE node
W20260620 23:08:43.600061 7706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.600471 7701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:43.600769 7701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:43.601924 7701 hybrid_clock.cc:648] HybridClock initialized: now 1781996923601904 us; error 33 us; skew 500 ppm
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:43.604349 7701 init.cc:377] Logged in from keytab as oryx/127.1.32.125@KRBTEST.COM (short username oryx)
I20260620 23:08:43.604804 7701 webserver.cc:460] Webserver started at http://127.1.32.125:41685/ using document root <none> and password file <none>
I20260620 23:08:43.605042 7701 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.605116 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.605265 7701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:43.606303 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/instance:
uuid: "28ac6218513c49908d70358f6972b6b3"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "93ff62d84e9cba5bbcdcc105b9cf27b6"
server_key_iv: "b29e922bf219911fb8d020c61f0c1315"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.606679 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal/instance:
uuid: "28ac6218513c49908d70358f6972b6b3"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "93ff62d84e9cba5bbcdcc105b9cf27b6"
server_key_iv: "b29e922bf219911fb8d020c61f0c1315"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.608100 7701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:43.608798 7716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.608968 7701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.609058 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
uuid: "28ac6218513c49908d70358f6972b6b3"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "93ff62d84e9cba5bbcdcc105b9cf27b6"
server_key_iv: "b29e922bf219911fb8d020c61f0c1315"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.609143 7701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.619391 7701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:43.621035 7701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:43.621184 7701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:43.624946 7701 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.125:35437
I20260620 23:08:43.625000 7768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.125:35437 every 8 connection(s)
I20260620 23:08:43.625349 7701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
I20260620 23:08:43.626246 7769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:43.627347 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 7701
I20260620 23:08:43.628185 7769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.125@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.32.125@KRBTEST.COM
I20260620 23:08:43.637985 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.629124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.1.32.126:35483)
0620 23:08:43.629187 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.629189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.629343 (+ 154us) server_negotiation.cc:407] Connection header received
0620 23:08:43.629405 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.629405 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.629414 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.629458 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.629949 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.630659 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.632009 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.632289 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.632366 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.634686 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.634688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.634688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.634696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.636467 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.636939 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.636940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.636941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.637002 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.637289 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.637290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.637290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.637461 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.637542 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.637806 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.637891 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:08:43.638206 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.628006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48366 (local address 127.1.32.125:35437)
0620 23:08:43.628178 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.628185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.628209 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:08:43.628673 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.628679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.628760 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.628927 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.629655 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.630843 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.631888 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.632314 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.632409 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.635333 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.635340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.635341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.635348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.637061 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.637389 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.637392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.637392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.637447 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.637642 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.637643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.637643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.637770 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.637917 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.637985 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.638058 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260620 23:08:43.638204 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.629071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:56818)
0620 23:08:43.629224 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.629277 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.629346 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.629486 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.629489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.629785 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.629870 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.629874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.630772 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.630773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.631946 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.631948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.632413 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.632414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.632491 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.633073 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.633083 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.634598 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.636549 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.636554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.636556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.636836 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.637079 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.637079 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.637080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.637194 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.637546 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.637553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.637675 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.637785 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.637950 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":104,"threads_started":1}
I20260620 23:08:43.638846 7769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.125@KRBTEST.COM for oryx/127.1.32.125@KRBTEST.COM
I20260620 23:08:43.644333 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.639156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.1.32.125:35437)
0620 23:08:43.639186 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.639187 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.639193 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:08:43.639262 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.639263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.639269 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.639311 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.639503 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.639921 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.640490 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.640658 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.640702 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.642218 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.642219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.642219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.642225 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.643341 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.643636 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.643637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.643637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.643679 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.643817 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.643817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.643817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.643904 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.643984 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.644119 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.644223 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:08:43.644404 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.639105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.0.0.1:48378)
0620 23:08:43.639153 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.639163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.639201 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.639313 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.639314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.639423 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.639453 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.639454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.639994 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.639995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.640422 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.640423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.640714 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.640715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.640758 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.641143 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.641148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.642151 (+ 1003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.643415 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.643416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.643416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.643567 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.643732 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.643733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.643733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.643779 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.643978 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.643979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.644011 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.644102 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.644289 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":89}
I20260620 23:08:43.645545 7769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Bootstrap starting.
I20260620 23:08:43.646315 7769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:43.646615 7769 log.cc:824] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:43.647370 7769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: No bootstrap required, opened a new log
I20260620 23:08:43.648675 7769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } }
I20260620 23:08:43.648795 7769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:43.648838 7769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ac6218513c49908d70358f6972b6b3, State: Initialized, Role: FOLLOWER
I20260620 23:08:43.648952 7769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } }
I20260620 23:08:43.649217 7775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } } }
I20260620 23:08:43.649255 7769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:43.649295 7775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:08:43.650849 7786 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:43.650915 7786 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:43.652216 7701 master_runner.cc:428] Detected that this master 28ac6218513c49908d70358f6972b6b3 is joining an existing cluster
I20260620 23:08:43.652271 7701 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.32.125:35437
I20260620 23:08:43.659927 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.653128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.1.32.126:35483)
0620 23:08:43.653170 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.653172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.653311 (+ 139us) server_negotiation.cc:407] Connection header received
0620 23:08:43.653351 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.653352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.653359 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.653401 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.654003 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.654435 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.655519 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.655788 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.655859 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.656785 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.656786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.656786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.656793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.658455 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.658855 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.658856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.658856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.658929 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.659127 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.659129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.659129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.659271 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.659332 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.659768 (+ 436us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.659840 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:08:43.660629 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.653084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:56824)
0620 23:08:43.653229 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.653237 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.653306 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.653408 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.653408 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.653576 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.653930 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.653931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.654506 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.654506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.655429 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.655431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.655880 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.655881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.655950 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.656358 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.656363 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.656699 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.658559 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.658559 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.658560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.658773 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.658986 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.658987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.658987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.659055 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.659325 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.659326 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.659387 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.659725 (+ 338us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.659832 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260620 23:08:43.661621 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.653090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.0.0.1:48392)
0620 23:08:43.655135 (+ 2045us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.655145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.655221 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.655327 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.655328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.655475 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.655550 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.655551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.656422 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.656423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.657193 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.657196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.657963 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.657964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.658031 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.658461 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.658466 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.658934 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.660548 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.660549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.660549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.660774 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.660965 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.660966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.660966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.661019 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.661299 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.661300 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.661350 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.661429 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.661531 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2029,"thread_start_us":79,"threads_started":1}
I20260620 23:08:43.661743 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.653153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.1.32.125:35437)
0620 23:08:43.653213 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.653214 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.655209 (+ 1995us) server_negotiation.cc:407] Connection header received
0620 23:08:43.655264 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.655264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.655273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.655323 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.655621 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.656300 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.657274 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.657836 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.657961 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.659044 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.659045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.659045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.659050 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.660468 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.660858 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.660858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.660859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.660904 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.661087 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.661088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.661088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.661213 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.661256 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.661458 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.661599 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":47}
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.126@KRBTEST.COM for oryx/127.1.32.125@KRBTEST.COM
I20260620 23:08:43.669538 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.662957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.1.32.125:35437)
0620 23:08:43.662989 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.662990 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.663122 (+ 132us) server_negotiation.cc:407] Connection header received
0620 23:08:43.663181 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.663181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.663188 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.663228 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.663640 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.664056 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.665045 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.665238 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.665288 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.667192 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.667193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.667193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.667199 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.668429 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.668801 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.668802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.668802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.668855 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.669047 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.669048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.669049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.669146 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.669216 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.669382 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.669449 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:43.669660 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.662854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.0.0.1:48394)
0620 23:08:43.663008 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.663048 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.663118 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.663233 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.663235 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.663529 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.663577 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.663579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.664126 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.664127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.664964 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.664967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.665309 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.665310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.665384 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.665774 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.665779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.667108 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.668526 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.668532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.668535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.668727 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.668920 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.668921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.668921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.668987 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.669210 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.669217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.669263 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.669359 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.669462 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260620 23:08:43.670122 7237 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.32.125:35437: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "28ac6218513c49908d70358f6972b6b3"
member_type: NON_VOTER
last_known_addr {
host: "127.1.32.125"
port: 35437
}
attrs {
promote: true
}
}
dest_uuid: "6733ba54101e4f608761e9288c8d5f4c"
cas_config_opid_index: -1
I20260620 23:08:43.670415 7237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } }
I20260620 23:08:43.670915 7795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 28ac6218513c49908d70358f6972b6b3 (127.1.32.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } }
I20260620 23:08:43.671142 7743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:43.671944 7795 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.32.125:35437
I20260620 23:08:43.671957 7743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Refusing update from remote peer 6733ba54101e4f608761e9288c8d5f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 23:08:43.672086 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } }
I20260620 23:08:43.672144 7701 master.cc:537] Master@127.1.32.125:35437 shutting down...
I20260620 23:08:43.672328 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:43.672451 7796 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:08:43.673064 7221 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260620 23:08:43.673471 7221 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c -> Peer 28ac6218513c49908d70358f6972b6b3 (127.1.32.125:35437): Couldn't send request to peer 28ac6218513c49908d70358f6972b6b3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 23:08:43.674913 7701 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:43.674989 7701 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:43.675033 7701 tablet_replica.cc:330] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: stopping tablet replica
I20260620 23:08:43.687767 7701 master.cc:552] Master@127.1.32.125:35437 shutdown complete.
I20260620 23:08:43.687803 7701 master_runner.cc:305] Clearing existing system tablet
I20260620 23:08:43.687938 7701 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.687973 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.688575 7701 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:43.689096 7800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.689204 7701 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.689253 7701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.689276 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
uuid: "28ac6218513c49908d70358f6972b6b3"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "93ff62d84e9cba5bbcdcc105b9cf27b6"
server_key_iv: "b29e922bf219911fb8d020c61f0c1315"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.689893 7701 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:43.691087 7701 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 23:08:43.691143 7701 log.cc:1196] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 23:08:43.691335 7701 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Deleting consensus metadata
I20260620 23:08:43.691457 7701 master_runner.cc:315] Copying system tablet from 127.1.32.126:35483
I20260620 23:08:43.692028 7701 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: tablet copy: Beginning tablet copy session from remote peer at address 127.1.32.126:35483
I20260620 23:08:43.697693 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.692347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.1.32.126:35483)
0620 23:08:43.692393 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.692396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.692454 (+ 58us) server_negotiation.cc:407] Connection header received
0620 23:08:43.692528 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.692529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.692539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.692598 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.692894 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.693379 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.694094 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.694288 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.694385 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.695350 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.695351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.695352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.695358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.696609 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.696942 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.696943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.696943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.696993 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.697201 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.697202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.697202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.697309 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.697382 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.697529 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.697609 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:08:43.697705 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.692287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:56834)
0620 23:08:43.692397 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.692407 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.692470 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.692597 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.692597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.692762 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.692809 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.692810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.693456 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.693457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.694000 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.694002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.694387 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.694387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.694479 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.694945 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.694950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.695245 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.696700 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.696701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.696701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.696876 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.697060 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.697060 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.697061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.697120 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.697374 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.697375 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.697417 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.697503 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.697600 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":63,"threads_started":1}
I20260620 23:08:43.698002 7257 tablet_copy_service.cc:140] P 6733ba54101e4f608761e9288c8d5f4c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 28ac6218513c49908d70358f6972b6b3 ({username='oryx', principal='oryx/127.1.32.125@KRBTEST.COM'} at 127.0.0.1:56834)
I20260620 23:08:43.698083 7257 tablet_copy_service.cc:161] P 6733ba54101e4f608761e9288c8d5f4c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 28ac6218513c49908d70358f6972b6b3 at {username='oryx', principal='oryx/127.1.32.125@KRBTEST.COM'} at 127.0.0.1:56834: session id = 28ac6218513c49908d70358f6972b6b3-00000000000000000000000000000000
I20260620 23:08:43.698650 7257 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 23:08:43.699213 7701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 23:08:43.699970 7701 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: tablet copy: Starting download of 0 data blocks...
I20260620 23:08:43.700099 7701 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: tablet copy: Starting download of 1 WAL segments...
I20260620 23:08:43.700822 7701 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 23:08:43.701542 7257 tablet_copy_service.cc:342] P 6733ba54101e4f608761e9288c8d5f4c: Request end of tablet copy session 28ac6218513c49908d70358f6972b6b3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.32.125@KRBTEST.COM'} at 127.0.0.1:56834
I20260620 23:08:43.701602 7257 tablet_copy_service.cc:433] P 6733ba54101e4f608761e9288c8d5f4c: ending tablet copy session 28ac6218513c49908d70358f6972b6b3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 28ac6218513c49908d70358f6972b6b3
I20260620 23:08:43.703909 7701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:43.704988 7810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.705034 7807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.705073 7808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.705305 7701 server_base.cc:1034] running on GCE node
I20260620 23:08:43.705405 7701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:43.705533 7701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:43.706647 7701 hybrid_clock.cc:648] HybridClock initialized: now 1781996923706639 us; error 22 us; skew 500 ppm
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:43.707779 7701 init.cc:377] Logged in from keytab as oryx/127.1.32.125@KRBTEST.COM (short username oryx)
I20260620 23:08:43.708051 7701 webserver.cc:460] Webserver started at http://127.1.32.125:36919/ using document root <none> and password file <none>
I20260620 23:08:43.708158 7701 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.708182 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.708688 7701 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.709125 7817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.709240 7701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.709303 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
uuid: "28ac6218513c49908d70358f6972b6b3"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "93ff62d84e9cba5bbcdcc105b9cf27b6"
server_key_iv: "b29e922bf219911fb8d020c61f0c1315"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.709353 7701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.713213 7701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:43.714546 7701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:43.717612 7701 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.125:35437
I20260620 23:08:43.717952 7869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.125:35437 every 8 connection(s)
I20260620 23:08:43.718024 7701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
I20260620 23:08:43.718740 7870 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:08:43.719169 7870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Bootstrap starting.
I20260620 23:08:43.721179 7870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:08:43.721387 7870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Bootstrap complete.
I20260620 23:08:43.721573 7870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } }
I20260620 23:08:43.721637 7870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 28ac6218513c49908d70358f6972b6b3, State: Initialized, Role: LEARNER
I20260620 23:08:43.721659 7870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } }
I20260620 23:08:43.721814 7870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:43.721825 7873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } }
I20260620 23:08:43.721905 7873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:08:43.723183 7884 catalog_manager.cc:1259] Loaded cluster ID: 3d778f7abbef4e43944afd7af343d278
I20260620 23:08:43.723260 7884 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: loading cluster ID for follower catalog manager: success
I20260620 23:08:43.724344 7884 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: acquiring CA information for follower catalog manager: success
I20260620 23:08:43.724615 7884 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:08:43.746353 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.739755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48410 (local address 127.1.32.125:35437)
0620 23:08:43.739885 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.739887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.739898 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:43.739923 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.739924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.739934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.740010 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.740504 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.741512 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.742158 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.742343 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.742414 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.743664 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.743665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.743666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.743672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.745252 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.745576 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.745577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.745577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.745634 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.745823 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.745823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.745824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.745927 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.746027 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.746152 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.746240 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for oryx/127.1.32.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.124.keytab.
Entry for principal oryx/127.1.32.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.124.keytab.
I20260620 23:08:43.769172 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:40303
--webserver_interface=127.1.32.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.124.keytab
--principal=oryx/127.1.32.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:43.847524 7888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:43.847712 7888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:43.847733 7888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:43.848785 7888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:43.848817 7888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:43.848831 7888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:43.848843 7888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:43.848855 7888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:43.850178 7888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.124.keytab
--principal=oryx/127.1.32.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:40303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.32.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:43.850346 7888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:43.850498 7888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:43.855408 7893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.855387 7894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.855392 7896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.855741 7888 server_base.cc:1034] running on GCE node
I20260620 23:08:43.855971 7888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:43.856251 7888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:43.857427 7888 hybrid_clock.cc:648] HybridClock initialized: now 1781996923857414 us; error 25 us; skew 500 ppm
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:43.859637 7888 init.cc:377] Logged in from keytab as oryx/127.1.32.124@KRBTEST.COM (short username oryx)
I20260620 23:08:43.860059 7888 webserver.cc:460] Webserver started at http://127.1.32.124:44049/ using document root <none> and password file <none>
I20260620 23:08:43.860275 7888 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.860340 7888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.860481 7888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:43.861419 7888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/instance:
uuid: "f3a030df78544205998dbcb4e5f19c05"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "26f2abe6c9ae20773f980acb393b2be2"
server_key_iv: "b25f0811051e9c66acb58fbb288ed790"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.861752 7888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal/instance:
uuid: "f3a030df78544205998dbcb4e5f19c05"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "26f2abe6c9ae20773f980acb393b2be2"
server_key_iv: "b25f0811051e9c66acb58fbb288ed790"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.863020 7888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:43.863708 7903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.863868 7888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:43.863958 7888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
uuid: "f3a030df78544205998dbcb4e5f19c05"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "26f2abe6c9ae20773f980acb393b2be2"
server_key_iv: "b25f0811051e9c66acb58fbb288ed790"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.864039 7888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.872134 7888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:43.873906 7888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:43.874053 7888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:43.877735 7888 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.124:40303
I20260620 23:08:43.877761 7955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.124:40303 every 8 connection(s)
I20260620 23:08:43.878067 7888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
I20260620 23:08:43.879062 7956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:43.880558 7956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:43.883556 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 7888
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.124@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.32.124@KRBTEST.COM
I20260620 23:08:43.889662 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.881272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.1.32.126:35483)
0620 23:08:43.881317 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.881319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.881409 (+ 90us) server_negotiation.cc:407] Connection header received
0620 23:08:43.881806 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.881807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.881815 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.881867 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.882503 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.883039 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.884254 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.884567 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.884635 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.886618 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.886620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.886620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.886626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.888243 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.888729 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.888730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.888731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.888791 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.889021 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.889022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.889022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.889160 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.889234 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.889462 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.889539 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:43.889788 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.881150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:56848)
0620 23:08:43.881297 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.881333 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.881732 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.881931 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.881943 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.882181 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.882375 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.882378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.883120 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.883121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.884171 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.884174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.884651 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.884652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.884743 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.885240 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.885248 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.886545 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.888394 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.888401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.888403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.888650 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.888863 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.888863 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.888863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.888958 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.889226 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.889230 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.889330 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.889410 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.889534 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260620 23:08:43.890604 7956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:08:43.892494 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.884041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.1.32.124:40303)
0620 23:08:43.884221 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.884225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.884242 (+ 17us) server_negotiation.cc:407] Connection header received
0620 23:08:43.884262 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.884264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.884287 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.884357 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.885015 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.885771 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.886471 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.886803 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.886861 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.889585 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.889590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.889590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.889596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.891368 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.891708 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.891711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.891711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.891762 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.891955 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.891955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.891956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.892085 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.892241 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.892316 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.892387 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.124@KRBTEST.COM for oryx/127.1.32.125@KRBTEST.COM
I20260620 23:08:43.896209 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.890826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.0.0.1:48422)
0620 23:08:43.890867 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.890876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.890927 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.891102 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.891102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.891284 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.891322 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.891323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.891915 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.891916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.892378 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.892380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.892860 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.892861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.892956 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.893260 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.893263 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.894258 (+ 995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.895539 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.895540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.895540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.895693 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.895838 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.895838 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.895839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.895877 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.896031 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.896031 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.896058 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.896106 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.896159 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 23:08:43.896256 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.890928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.1.32.125:35437)
0620 23:08:43.890976 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.890978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.890985 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:43.891009 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.891010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.891021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.891098 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.891397 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.891841 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.892476 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.892775 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.892861 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.894327 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.894328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.894329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.894334 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.895473 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.895752 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.895752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.895753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.895796 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.895907 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.895907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.895908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.895998 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.896040 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.896123 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.896180 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:08:43.896549 7956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.124@KRBTEST.COM for oryx/127.1.32.124@KRBTEST.COM
I20260620 23:08:43.901609 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.896743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.0.0.1:49048)
0620 23:08:43.896788 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.896798 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.896829 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.896910 (+ 81us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.896910 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.897006 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.897027 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.897028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.897489 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.897490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.897877 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.897878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.898162 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.898163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.898192 (+ 29us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.898459 (+ 267us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.898462 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.899391 (+ 929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.900825 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.900826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.900826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.900986 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.901172 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.901173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.901173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.901227 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.901410 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.901411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.901445 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.901492 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.901559 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 23:08:43.901678 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.896790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49048 (local address 127.1.32.124:40303)
0620 23:08:43.896835 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.896836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.896843 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:43.896861 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.896861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.896868 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.896908 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.897062 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.897438 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.897935 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.898109 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.898153 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.899487 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.899488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.899488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.899494 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.900746 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.901048 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.901048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.901049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.901098 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.901282 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.901282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.901283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.901361 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.901404 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.901505 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.901601 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":31}
I20260620 23:08:43.902696 7956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Bootstrap starting.
I20260620 23:08:43.903479 7956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:43.903784 7956 log.cc:824] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:43.904398 7956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: No bootstrap required, opened a new log
I20260620 23:08:43.905740 7956 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } }
I20260620 23:08:43.905861 7956 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:43.905903 7956 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a030df78544205998dbcb4e5f19c05, State: Initialized, Role: FOLLOWER
I20260620 23:08:43.906041 7956 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } }
I20260620 23:08:43.906257 7961 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } } }
I20260620 23:08:43.906304 7956 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:43.906347 7961 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:08:43.907903 7972 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:08:43.907982 7972 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:08:43.910231 7888 master_runner.cc:428] Detected that this master f3a030df78544205998dbcb4e5f19c05 is joining an existing cluster
I20260620 23:08:43.910302 7888 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.32.124:40303
I20260620 23:08:43.918406 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.911321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.0.0.1:48438)
0620 23:08:43.911583 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.911593 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.911672 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.911874 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.911875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.912044 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.912093 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.912094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.913022 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.913023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.913858 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.913861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.914312 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.914313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.914371 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.914823 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.914827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.915258 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.917266 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.917267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.917267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.917464 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.917685 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.917686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.917686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.917763 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.918087 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.918088 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.918127 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.918199 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.918308 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":25,"thread_start_us":67,"threads_started":1}
I20260620 23:08:43.918435 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.911477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.1.32.125:35437)
0620 23:08:43.911511 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.911513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.911647 (+ 134us) server_negotiation.cc:407] Connection header received
0620 23:08:43.911765 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.911766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.911776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.911808 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.912201 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.912895 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.913939 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.914201 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.914256 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.915615 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.915617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.915617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.915624 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.917175 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.917543 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.917545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.917545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.917609 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.917871 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.917872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.917872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.918011 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.918089 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.918241 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.918318 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:43.919296 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.911505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.1.32.126:35483)
0620 23:08:43.911546 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.911548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.911635 (+ 87us) server_negotiation.cc:407] Connection header received
0620 23:08:43.911694 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.911695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.911705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.911765 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.912166 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.912885 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.914679 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.914997 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.915038 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.916419 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.916420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.916421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.916428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.918060 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.918505 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.918506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.918506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.918563 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.918767 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.918768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.918768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.918916 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.918985 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.919166 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.919237 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:08:43.919299 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.911308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:56850)
0620 23:08:43.911580 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.911589 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.911645 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.911776 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.911777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.912012 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.912080 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.912081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.912988 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.912989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.914588 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.914590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.915105 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.915106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.915164 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.915648 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.915652 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.916333 (+ 681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.918145 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.918146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.918146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.918408 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.918640 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.918641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.918641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.918701 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.919013 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.919014 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.919051 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.919115 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.919204 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":87,"threads_started":1}
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996922, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.126@KRBTEST.COM for oryx/127.1.32.124@KRBTEST.COM
I20260620 23:08:43.938022 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.921288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49058 (local address 127.1.32.124:40303)
0620 23:08:43.921429 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.921430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.921438 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:43.921475 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.921475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.921484 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.921554 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.921896 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.922553 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.923489 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.923704 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.923786 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.925503 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.925505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.925505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.925512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.936774 (+ 11262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.937104 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.937105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.937106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.937165 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.937453 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.937454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.937455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.937584 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.937676 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.937831 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.937915 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260620 23:08:43.938017 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.921236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.0.0.1:49058)
0620 23:08:43.921290 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.921299 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.921337 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.921572 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.921591 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.921776 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.921822 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.921824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.922638 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.922639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.923392 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.923395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.923788 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.923789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.923854 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.924247 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.924251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.925413 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.936843 (+ 11430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.936844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.936845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.937037 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.937307 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.937309 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.937309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.937381 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.937659 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.937660 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.937713 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.937789 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.937902 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 23:08:43.938810 7237 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.32.124:40303: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f3a030df78544205998dbcb4e5f19c05"
member_type: NON_VOTER
last_known_addr {
host: "127.1.32.124"
port: 40303
}
attrs {
promote: true
}
}
dest_uuid: "6733ba54101e4f608761e9288c8d5f4c"
cas_config_opid_index: 5
I20260620 23:08:43.939031 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.911371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.0.0.1:49054)
0620 23:08:43.913194 (+ 1823us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.913202 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.913270 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.913403 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.913404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.913559 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.913597 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.913598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.914273 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.914274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.915091 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.915094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.915529 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.915530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.915589 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.935378 (+ 19789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.935382 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.935812 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.937840 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.937841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.937841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.938045 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.938233 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.938234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.938234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.938279 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.938635 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.938635 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.938714 (+ 79us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.938789 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.938913 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1809,"thread_start_us":61,"threads_started":1}
I20260620 23:08:43.939093 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.911505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.1.32.124:40303)
0620 23:08:43.911576 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.911578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.913259 (+ 1681us) server_negotiation.cc:407] Connection header received
0620 23:08:43.913337 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.913338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.913346 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.913400 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.913666 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.914188 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.915184 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.915454 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.915529 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.935921 (+ 20392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.935923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.935923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.935930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.937767 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.938123 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.938124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.938125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.938174 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.938339 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.938339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.938339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.938437 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.938472 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.938822 (+ 350us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.938965 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":55}
I20260620 23:08:43.939071 7237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } }
I20260620 23:08:43.939430 7796 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f3a030df78544205998dbcb4e5f19c05 (127.1.32.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.939769 7930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:43.940292 7796 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.32.124:40303
I20260620 23:08:43.940372 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.940445 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:43.940508 7888 master.cc:537] Master@127.1.32.124:40303 shutting down...
I20260620 23:08:43.940634 7930 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Refusing update from remote peer 6733ba54101e4f608761e9288c8d5f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 23:08:43.941138 7795 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:08:43.941752 7224 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c -> Peer f3a030df78544205998dbcb4e5f19c05 (127.1.32.124:40303): Couldn't send request to peer f3a030df78544205998dbcb4e5f19c05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 23:08:43.945067 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.939477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.0.0.1:48446)
0620 23:08:43.939633 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.939643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.939678 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.941574 (+ 1896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.941575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.941577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:43.941610 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.941612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.942368 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.942369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.944050 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.944053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.944850 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.944851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.944907 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.944930 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.944974 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.945005 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260620 23:08:43.945448 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.939533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.1.32.125:35437)
0620 23:08:43.941394 (+ 1861us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.941396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.941404 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:43.941427 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.941428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.941450 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.941520 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:43.941673 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.942278 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.944175 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.944688 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.944750 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.944816 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.945311 (+ 495us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.945359 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1839}
I20260620 23:08:43.946003 7844 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 LEARNER]: Refusing update from remote peer 6733ba54101e4f608761e9288c8d5f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 23:08:43.946215 7795 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 23:08:43.946821 7888 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:43.946753 7844 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f3a030df78544205998dbcb4e5f19c05 (127.1.32.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.946995 7888 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:43.947049 7888 tablet_replica.cc:330] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: stopping tablet replica
I20260620 23:08:43.947887 7873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6733ba54101e4f608761e9288c8d5f4c. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.947916 7983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: NON_VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: true } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.947974 7873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:08:43.947976 7983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:08:43.948498 7796 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: attempting to promote NON_VOTER 28ac6218513c49908d70358f6972b6b3 to VOTER
I20260620 23:08:43.948685 7796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } }
I20260620 23:08:43.949170 7844 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 LEARNER]: Refusing update from remote peer 6733ba54101e4f608761e9288c8d5f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260620 23:08:43.949352 7224 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c -> Peer f3a030df78544205998dbcb4e5f19c05 (127.1.32.124:40303): Couldn't send request to peer f3a030df78544205998dbcb4e5f19c05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 23:08:43.949398 7796 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 23:08:43.949898 7796 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 28ac6218513c49908d70358f6972b6b3 (127.1.32.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.950929 7795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.950990 7795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:43.950786 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.951179 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:43.951215 7844 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 28ac6218513c49908d70358f6972b6b3 (127.1.32.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.951273 7888 master.cc:552] Master@127.1.32.124:40303 shutdown complete.
I20260620 23:08:43.951352 7888 master_runner.cc:305] Clearing existing system tablet
I20260620 23:08:43.951490 7888 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.951537 7888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.952005 7983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.952111 7983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:43.952179 7888 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:43.952644 7986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.952761 7888 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.952821 7888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.952852 7888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
uuid: "f3a030df78544205998dbcb4e5f19c05"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "26f2abe6c9ae20773f980acb393b2be2"
server_key_iv: "b25f0811051e9c66acb58fbb288ed790"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.953403 7888 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:43.954380 7888 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 23:08:43.954463 7888 log.cc:1196] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 23:08:43.954653 7888 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Deleting consensus metadata
I20260620 23:08:43.954813 7888 master_runner.cc:315] Copying system tablet from 127.1.32.126:35483
I20260620 23:08:43.955363 7888 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: tablet copy: Beginning tablet copy session from remote peer at address 127.1.32.126:35483
I20260620 23:08:43.961532 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.955673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.1.32.126:35483)
0620 23:08:43.955711 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:43.955713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:43.955796 (+ 83us) server_negotiation.cc:407] Connection header received
0620 23:08:43.955857 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:43.955858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:43.955865 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:43.955912 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:43.956222 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.956758 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.957802 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.957968 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.958020 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.958972 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:43.958973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:43.958974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:43.958980 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:43.960500 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.960864 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.960864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.960865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.960913 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:43.961098 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:43.961098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:43.961098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:43.961206 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:43.961257 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:43.961396 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 23:08:43.961479 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:08:43.961565 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:43.955626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:56864)
0620 23:08:43.955742 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:43.955751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:43.955803 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:43.955917 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:43.955917 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:43.956103 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:43.956159 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.956160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.956841 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.956842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.957733 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:43.957736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:43.958039 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:43.958040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:43.958103 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:43.958495 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:43.958500 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:43.958864 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:43.960584 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.960585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.960585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.960795 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.960983 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:43.960984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:43.960984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:43.961043 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:43.961266 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:43.961266 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:43.961307 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 23:08:43.961380 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:08:43.961472 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260620 23:08:43.962054 7257 tablet_copy_service.cc:140] P 6733ba54101e4f608761e9288c8d5f4c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f3a030df78544205998dbcb4e5f19c05 ({username='oryx', principal='oryx/127.1.32.124@KRBTEST.COM'} at 127.0.0.1:56864)
I20260620 23:08:43.962136 7257 tablet_copy_service.cc:161] P 6733ba54101e4f608761e9288c8d5f4c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f3a030df78544205998dbcb4e5f19c05 at {username='oryx', principal='oryx/127.1.32.124@KRBTEST.COM'} at 127.0.0.1:56864: session id = f3a030df78544205998dbcb4e5f19c05-00000000000000000000000000000000
I20260620 23:08:43.962558 7257 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 23:08:43.963047 7888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 23:08:43.963900 7888 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: tablet copy: Starting download of 0 data blocks...
I20260620 23:08:43.964058 7888 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: tablet copy: Starting download of 1 WAL segments...
I20260620 23:08:43.965009 7888 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 23:08:43.965802 7257 tablet_copy_service.cc:342] P 6733ba54101e4f608761e9288c8d5f4c: Request end of tablet copy session f3a030df78544205998dbcb4e5f19c05-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.32.124@KRBTEST.COM'} at 127.0.0.1:56864
I20260620 23:08:43.965857 7257 tablet_copy_service.cc:433] P 6733ba54101e4f608761e9288c8d5f4c: ending tablet copy session f3a030df78544205998dbcb4e5f19c05-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f3a030df78544205998dbcb4e5f19c05
I20260620 23:08:43.968444 7888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:43.969609 7994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.969766 7993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:43.969650 7996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:43.971035 7888 server_base.cc:1034] running on GCE node
I20260620 23:08:43.971133 7888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:43.971263 7888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:43.972388 7888 hybrid_clock.cc:648] HybridClock initialized: now 1781996923972380 us; error 23 us; skew 500 ppm
Jun 20 23:08:43 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996923, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:43.973654 7888 init.cc:377] Logged in from keytab as oryx/127.1.32.124@KRBTEST.COM (short username oryx)
I20260620 23:08:43.973939 7888 webserver.cc:460] Webserver started at http://127.1.32.124:35349/ using document root <none> and password file <none>
I20260620 23:08:43.974053 7888 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:43.974099 7888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:43.974721 7888 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:43.975322 8003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.975476 7888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:43.975529 7888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
uuid: "f3a030df78544205998dbcb4e5f19c05"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "26f2abe6c9ae20773f980acb393b2be2"
server_key_iv: "b25f0811051e9c66acb58fbb288ed790"
server_key_version: "encryptionkey@0"
I20260620 23:08:43.975586 7888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:43.983153 7888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:43.984576 7888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:43.987491 7888 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.124:40303
I20260620 23:08:43.987507 8055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.124:40303 every 8 connection(s)
I20260620 23:08:43.987785 7888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
I20260620 23:08:43.988391 8056 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:08:43.988749 8056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Bootstrap starting.
I20260620 23:08:43.990828 8056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:08:43.991060 8056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: Bootstrap complete.
I20260620 23:08:43.991231 8056 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } }
I20260620 23:08:43.991308 8056 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f3a030df78544205998dbcb4e5f19c05, State: Initialized, Role: LEARNER
I20260620 23:08:43.991354 8056 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } }
I20260620 23:08:43.991554 8056 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:43.991531 8059 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:43.991598 8059 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: This master's current role is: LEARNER
I20260620 23:08:43.992924 8070 catalog_manager.cc:1259] Loaded cluster ID: 3d778f7abbef4e43944afd7af343d278
I20260620 23:08:43.992990 8070 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: loading cluster ID for follower catalog manager: success
I20260620 23:08:43.993983 8070 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: acquiring CA information for follower catalog manager: success
I20260620 23:08:43.994232 8070 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:08:44.006000 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.000016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49068 (local address 127.1.32.124:40303)
0620 23:08:44.000116 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.000118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.000127 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:44.000148 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.000149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.000158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.000233 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.000789 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.001380 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.002041 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.002232 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.002322 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.003515 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.003516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.003516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.003522 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.004886 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.005269 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.005288 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.005289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.005336 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.005558 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.005559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.005559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.005651 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.005725 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.005870 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.005925 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":49,"threads_started":1}
I20260620 23:08:44.006659 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 7299
I20260620 23:08:44.014794 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:37481
--local_ip_for_outbound_sockets=127.1.32.65
--tserver_master_addrs=127.1.32.126:35483
--webserver_port=35633
--webserver_interface=127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab
--principal=oryx/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:44.093860 8072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:44.094049 8072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:44.094074 8072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:44.095278 8072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:44.095317 8072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:44.095343 8072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:44.096776 8072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab
--principal=oryx/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:37481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=35633
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:44.097003 8072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:44.097244 8072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:44.101967 8078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:44.101995 8080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:44.102095 8072 server_base.cc:1034] running on GCE node
W20260620 23:08:44.101984 8077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:44.102366 8072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:44.102562 8072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:44.103691 8072 hybrid_clock.cc:648] HybridClock initialized: now 1781996924103685 us; error 28 us; skew 500 ppm
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:44.106103 8072 init.cc:377] Logged in from keytab as oryx/127.1.32.65@KRBTEST.COM (short username oryx)
I20260620 23:08:44.106498 8072 webserver.cc:460] Webserver started at http://127.1.32.65:35633/ using document root <none> and password file <none>
I20260620 23:08:44.106706 8072 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:44.106781 8072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:44.108004 8072 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:44.108503 8087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.108659 8072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:44.108752 8072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "83163b27b5d94c869492f1391b362a24"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "cb4fcbc9953fd62f1954dc8931218ec5"
server_key_iv: "d4d144ec830d90b5bdfd6429107082f3"
server_key_version: "encryptionkey@0"
I20260620 23:08:44.109071 8072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:44.119392 8072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:44.120973 8072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:44.121145 8072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:44.121343 8072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:44.121605 8072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:44.121654 8072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.121688 8072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:44.121714 8072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.128044 8072 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:37481
I20260620 23:08:44.128073 8200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:37481 every 8 connection(s)
I20260620 23:08:44.128417 8072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:44.129218 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 8072
I20260620 23:08:44.129379 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 7433
I20260620 23:08:44.135131 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:35447
--local_ip_for_outbound_sockets=127.1.32.66
--tserver_master_addrs=127.1.32.126:35483
--webserver_port=36957
--webserver_interface=127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab
--principal=oryx/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.124@KRBTEST.COM
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.125@KRBTEST.COM
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:44.141294 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.129317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:45737 (local address 127.1.32.124:40303)
0620 23:08:44.129449 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.129451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.129462 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:44.130159 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.130160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.130173 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.130203 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.131424 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.132251 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.133792 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.134116 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.134158 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.136872 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.136874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.136874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.136882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.139215 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140149 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.140151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.140151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.140218 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140489 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.140491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.140491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.140641 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.140694 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.141040 (+ 346us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.141150 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260620 23:08:44.141345 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.129162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:46883 (local address 127.1.32.126:35483)
0620 23:08:44.129214 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.129216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.132548 (+ 3332us) server_negotiation.cc:407] Connection header received
0620 23:08:44.132662 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.132663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.132677 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.132731 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.133135 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.133802 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.135816 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.136126 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.136203 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.138340 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.138342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.138342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.138350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.140216 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140584 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.140585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.140585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.140637 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140829 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.140829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.140830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.140946 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.140987 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.141196 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.141267 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:08:44.141367 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.129051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.1.32.65:57513)
0620 23:08:44.129508 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.129516 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.130039 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.130375 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.130379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.131011 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.131287 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.131293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.132343 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.132344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.133695 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.133698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.134209 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.134210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.134338 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.134932 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.134942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.137441 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.139606 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.139607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.139607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.140051 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.140268 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.140269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.140392 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.140704 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.140711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.140864 (+ 153us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.140982 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.141147 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":75,"thread_start_us":64,"threads_started":1}
I20260620 23:08:44.141631 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.129051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:46883)
0620 23:08:44.132483 (+ 3432us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.132493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.132574 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.132788 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.132789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.132972 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.133029 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.133031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.133892 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.133892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.135745 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.135748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.136213 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.136214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.136281 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.136937 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.136941 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.138232 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.140293 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.140294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.140507 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.140710 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.140710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.140762 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.141033 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.141034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.141082 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.141437 (+ 355us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.141531 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3402,"mutex_wait_us":301,"thread_start_us":71,"threads_started":1}
I20260620 23:08:44.141764 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.129379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57513 (local address 127.1.32.125:35437)
0620 23:08:44.129500 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.129501 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.129559 (+ 58us) server_negotiation.cc:407] Connection header received
0620 23:08:44.130156 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.130158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.130170 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.130203 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.131406 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.132207 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.133798 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.134074 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.134119 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.137596 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.137597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.137597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.137605 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.139521 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140138 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.140139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.140140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.140199 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140468 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.140469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.140469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.140617 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.140678 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.141588 (+ 910us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.141659 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260620 23:08:44.141770 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.129054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.1.32.65:45737)
0620 23:08:44.129335 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.129376 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.130039 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.130375 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.130379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.131253 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.131325 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.131328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.132368 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.132368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.133699 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.133701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.134232 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.134233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.134340 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.135079 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.135084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.136766 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.139564 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.139571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.139573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.140050 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.140298 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.140298 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.140299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.140397 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.140750 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.140752 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.140865 (+ 113us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.141500 (+ 635us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.141622 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":50,"thread_start_us":76,"threads_started":1}
I20260620 23:08:44.142179 8202 heartbeater.cc:347] Connected to a master server at 127.1.32.126:35483
I20260620 23:08:44.142270 8202 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.142455 8203 heartbeater.cc:347] Connected to a master server at 127.1.32.125:35437
I20260620 23:08:44.142531 8203 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.142463 8202 heartbeater.cc:511] Master 127.1.32.126:35483 requested a full tablet report, sending...
I20260620 23:08:44.142580 8201 heartbeater.cc:347] Connected to a master server at 127.1.32.124:40303
I20260620 23:08:44.142598 8203 heartbeater.cc:511] Master 127.1.32.125:35437 requested a full tablet report, sending...
I20260620 23:08:44.142611 8201 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.142663 8201 heartbeater.cc:511] Master 127.1.32.124:40303 requested a full tablet report, sending...
I20260620 23:08:44.143014 7237 ts_manager.cc:194] Re-registered known tserver with Master: 83163b27b5d94c869492f1391b362a24 (127.1.32.65:37481)
I20260620 23:08:44.143071 7834 ts_manager.cc:194] Registered new tserver with Master: 83163b27b5d94c869492f1391b362a24 (127.1.32.65:37481)
I20260620 23:08:44.143101 8020 ts_manager.cc:194] Registered new tserver with Master: 83163b27b5d94c869492f1391b362a24 (127.1.32.65:37481)
I20260620 23:08:44.143543 7237 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:46883
W20260620 23:08:44.236388 8208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:44.236567 8208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:44.236594 8208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:44.237684 8208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:44.237720 8208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:44.237744 8208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:44.239292 8208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab
--principal=oryx/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:35447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=36957
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:44.239509 8208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:44.239739 8208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:08:44.244405 8208 server_base.cc:1034] running on GCE node
W20260620 23:08:44.244366 8216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:44.244366 8213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:44.244457 8214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:44.244738 8208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:44.244974 8208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:44.246110 8208 hybrid_clock.cc:648] HybridClock initialized: now 1781996924246109 us; error 33 us; skew 500 ppm
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:44.248570 8208 init.cc:377] Logged in from keytab as oryx/127.1.32.66@KRBTEST.COM (short username oryx)
I20260620 23:08:44.248992 8208 webserver.cc:460] Webserver started at http://127.1.32.66:36957/ using document root <none> and password file <none>
I20260620 23:08:44.249202 8208 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:44.249277 8208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:44.250487 8208 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:44.251078 8223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.251298 8208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:08:44.251392 8208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "8e97115bc9b541b8962c6a974d6f90d6"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "a8076c12408fd13f971637d5ca538bee"
server_key_iv: "9e8ba1884df59d6fb654b9f247e084ab"
server_key_version: "encryptionkey@0"
I20260620 23:08:44.251709 8208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:44.261700 8208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:44.263918 8208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:44.264070 8208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:44.264333 8208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:44.264616 8208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:44.264655 8208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.264678 8208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:44.264720 8208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.270439 8208 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:35447
I20260620 23:08:44.270479 8336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:35447 every 8 connection(s)
I20260620 23:08:44.270787 8208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:44.272490 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 8208
I20260620 23:08:44.272660 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 7567
I20260620 23:08:44.278877 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:43369
--local_ip_for_outbound_sockets=127.1.32.67
--tserver_master_addrs=127.1.32.126:35483
--webserver_port=45801
--webserver_interface=127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab
--principal=oryx/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:32885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.125@KRBTEST.COM
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.124@KRBTEST.COM
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:44.283891 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.271735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:53219 (local address 127.1.32.125:35437)
0620 23:08:44.271783 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.271785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.271797 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:08:44.272409 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.272410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.272424 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.272456 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.273338 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.274098 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.275573 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.275958 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.276046 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.279615 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.279616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.279617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.279624 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.281673 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.282104 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.282105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.282105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.282157 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.282966 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.282967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.282968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.283138 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.283194 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.283714 (+ 520us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.283789 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:08:44.284511 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.271505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.1.32.66:53219)
0620 23:08:44.271661 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.271701 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.272288 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.272651 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.272655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.273017 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.273248 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.273253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.274223 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.274224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.275487 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.275489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.276050 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.276051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.276182 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.277515 (+ 1333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.277525 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.279526 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.281748 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.281754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.281756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.282027 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.282243 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.282244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.282244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.282383 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.283371 (+ 988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.283378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.283557 (+ 179us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.283693 (+ 136us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.284311 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260620 23:08:44.284718 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.271600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:40347 (local address 127.1.32.126:35483)
0620 23:08:44.271654 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.271656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.271943 (+ 287us) server_negotiation.cc:407] Connection header received
0620 23:08:44.272410 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.272412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.272424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.272456 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.274938 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.275669 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.276581 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.276901 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.276960 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.280948 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.280950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.280950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.280958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.283256 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.283774 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.283776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.283776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.283835 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.284038 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.284039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.284039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.284193 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.284273 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.284504 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.284588 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:08:44.284936 8339 heartbeater.cc:347] Connected to a master server at 127.1.32.125:35437
I20260620 23:08:44.285032 8339 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.285159 8343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.271505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:40347)
0620 23:08:44.271855 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.271864 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.272289 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.272651 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.272655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.273456 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.273515 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.273517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.275759 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.275760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.276499 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.276501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.276975 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.276976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.277055 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.278303 (+ 1248us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.278307 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.280710 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.283352 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.283353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.283354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.283671 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.283907 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.283907 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.283908 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.283963 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.284324 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.284325 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.284369 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.284835 (+ 466us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.284962 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260620 23:08:44.285316 8339 heartbeater.cc:511] Master 127.1.32.125:35437 requested a full tablet report, sending...
I20260620 23:08:44.285490 8338 heartbeater.cc:347] Connected to a master server at 127.1.32.126:35483
I20260620 23:08:44.285533 8338 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.285604 8338 heartbeater.cc:511] Master 127.1.32.126:35483 requested a full tablet report, sending...
I20260620 23:08:44.285745 7834 ts_manager.cc:194] Registered new tserver with Master: 8e97115bc9b541b8962c6a974d6f90d6 (127.1.32.66:35447)
I20260620 23:08:44.285828 7237 ts_manager.cc:194] Re-registered known tserver with Master: 8e97115bc9b541b8962c6a974d6f90d6 (127.1.32.66:35447)
I20260620 23:08:44.286339 7237 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:40347
I20260620 23:08:44.286948 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.271733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:53269 (local address 127.1.32.124:40303)
0620 23:08:44.271781 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.271783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.272013 (+ 230us) server_negotiation.cc:407] Connection header received
0620 23:08:44.272458 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.272460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.272474 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.272511 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.273705 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.274453 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.277187 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.277465 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.277523 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.280556 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.280557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.280557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.280564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.283377 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.286076 (+ 2699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.286077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.286078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.286141 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.286360 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.286361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.286362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.286497 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.286538 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.286727 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.286809 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:08:44.287139 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.271505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.1.32.66:53269)
0620 23:08:44.271879 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.271898 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.272311 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.273017 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.273018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.273582 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.273626 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.273627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.274541 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.274541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.277111 (+ 2570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.277113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.277575 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.277576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.277648 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.277992 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.277997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.280463 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.283458 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.283459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.283459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.284167 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.286219 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.286220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.286221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.286279 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.286584 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.286585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.286624 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.286694 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.286781 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":217,"thread_start_us":70,"threads_started":1}
I20260620 23:08:44.287533 8337 heartbeater.cc:347] Connected to a master server at 127.1.32.124:40303
I20260620 23:08:44.287611 8337 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.287693 8337 heartbeater.cc:511] Master 127.1.32.124:40303 requested a full tablet report, sending...
I20260620 23:08:44.287966 8020 ts_manager.cc:194] Registered new tserver with Master: 8e97115bc9b541b8962c6a974d6f90d6 (127.1.32.66:35447)
W20260620 23:08:44.359333 8344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:44.359546 8344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:44.359580 8344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:44.360682 8344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:44.360743 8344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:44.360787 8344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:44.362196 8344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:32885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab
--principal=oryx/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:43369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=45801
--tserver_master_addrs=127.1.32.126:35483,127.1.32.125:35437,127.1.32.124:40303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:44.362429 8344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:44.362632 8344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:44.367575 8352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:44.367677 8344 server_base.cc:1034] running on GCE node
W20260620 23:08:44.367571 8349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:44.367660 8350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:44.368010 8344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:44.368269 8344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:44.369484 8344 hybrid_clock.cc:648] HybridClock initialized: now 1781996924369436 us; error 59 us; skew 500 ppm
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:44.371814 8344 init.cc:377] Logged in from keytab as oryx/127.1.32.67@KRBTEST.COM (short username oryx)
I20260620 23:08:44.372195 8344 webserver.cc:460] Webserver started at http://127.1.32.67:45801/ using document root <none> and password file <none>
I20260620 23:08:44.372400 8344 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:44.372483 8344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:44.373731 8344 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:08:44.374307 8359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.374500 8344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:44.374603 8344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "ab2ac687bc4843c095f4a4bc474d289b"
format_stamp: "Formatted at 2026-06-20 23:08:43 on dist-test-slave-fzjm"
server_key: "19b8761241bce0991923742e32caaa00"
server_key_iv: "4857c9f2365662d8543f47ab2b57d8e3"
server_key_version: "encryptionkey@0"
I20260620 23:08:44.374939 8344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:44.387293 8344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:44.388898 8344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:44.389045 8344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:44.389235 8344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:44.389523 8344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:44.389580 8344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.389614 8344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:44.389640 8344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:44.395637 8344 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:43369
I20260620 23:08:44.395689 8472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:43369 every 8 connection(s)
I20260620 23:08:44.395977 8344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.125@KRBTEST.COM
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
Jun 20 23:08:44 dist-test-slave-fzjm krb5kdc[7192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996924, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.124@KRBTEST.COM
I20260620 23:08:44.404668 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 8344
I20260620 23:08:44.406744 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.396661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:53239 (local address 127.1.32.126:35483)
0620 23:08:44.396875 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.396877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.396952 (+ 75us) server_negotiation.cc:407] Connection header received
0620 23:08:44.397883 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.397884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.397896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.397971 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.398659 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.399387 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400482 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.400686 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400716 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.402967 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.402968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.402969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.402976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.404941 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.405521 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.405522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.405523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.405586 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.405880 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.405881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.405881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.406038 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.406105 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.406588 (+ 483us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.406665 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260620 23:08:44.406844 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.396712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:52621 (local address 127.1.32.125:35437)
0620 23:08:44.396752 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.396754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.396942 (+ 188us) server_negotiation.cc:407] Connection header received
0620 23:08:44.398072 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.398073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.398082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.398135 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.398659 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.399173 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400106 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.400310 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400389 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.403405 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.403406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.403406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.403414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.405006 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.405537 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.405538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.405539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.405599 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.406270 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.406271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.406271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.406417 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.406472 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.406670 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.406737 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:08:44.407454 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.396817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:43965 (local address 127.1.32.124:40303)
0620 23:08:44.396880 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.396882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.397018 (+ 136us) server_negotiation.cc:407] Connection header received
0620 23:08:44.397960 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.397962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.397972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.398020 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.398659 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.399390 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400494 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.400791 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400878 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.403483 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.403485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.403485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.403491 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.406325 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.406689 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.406690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.406690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.406740 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.406936 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.406948 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.406948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.407077 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.407136 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.407341 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.407395 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:08:44.407925 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.396598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:53239)
0620 23:08:44.396830 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.396867 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.397770 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.398058 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.398062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.398390 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.398559 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.398564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.399481 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.399482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.400373 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.400375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.400809 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.400868 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.401246 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.401250 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.402838 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.405046 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.405051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.405053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.405435 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.405673 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.405674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.405675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.405803 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.406096 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.406101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.406258 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.406376 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.407707 (+ 1331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":43,"thread_start_us":84,"threads_started":1}
I20260620 23:08:44.407928 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.396598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.1.32.67:43965)
0620 23:08:44.396947 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.396956 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.397774 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.398058 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.398062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.398383 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.398559 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.398564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.399481 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.399482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.400373 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.400375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.400879 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.400945 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.401416 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.401420 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.403414 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.406412 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.406413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.406414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.406622 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.406803 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.406804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.406804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.406852 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.407162 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.407163 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.407208 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.407270 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.407794 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":507,"spinlock_wait_cycles":384,"thread_start_us":93,"threads_started":1}
I20260620 23:08:44.408411 8473 heartbeater.cc:347] Connected to a master server at 127.1.32.124:40303
I20260620 23:08:44.408511 8473 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.408712 8473 heartbeater.cc:511] Master 127.1.32.124:40303 requested a full tablet report, sending...
I20260620 23:08:44.408857 8474 heartbeater.cc:347] Connected to a master server at 127.1.32.126:35483
I20260620 23:08:44.408923 8474 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.408993 8474 heartbeater.cc:511] Master 127.1.32.126:35483 requested a full tablet report, sending...
I20260620 23:08:44.409155 8020 ts_manager.cc:194] Registered new tserver with Master: ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369)
I20260620 23:08:44.409212 7237 ts_manager.cc:194] Re-registered known tserver with Master: ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369)
I20260620 23:08:44.409586 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.396598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.1.32.67:52621)
0620 23:08:44.396832 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.396867 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.397769 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.398152 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.398153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.398380 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:44.398559 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.398563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.399289 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.399290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.400019 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.400021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.400397 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.400398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.400569 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.401075 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:44.401082 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:44.403336 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:44.405110 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.405111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.405112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.405435 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.405972 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:44.405973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:44.405973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:44.406034 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:44.406516 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:44.406517 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:44.406558 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.407048 (+ 490us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.407935 (+ 887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":620,"thread_start_us":93,"threads_started":1}
I20260620 23:08:44.409703 7237 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:53239
I20260620 23:08:44.409883 8475 heartbeater.cc:347] Connected to a master server at 127.1.32.125:35437
I20260620 23:08:44.409950 8475 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:44.410017 8475 heartbeater.cc:511] Master 127.1.32.125:35437 requested a full tablet report, sending...
I20260620 23:08:44.410295 7834 ts_manager.cc:194] Registered new tserver with Master: ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369)
I20260620 23:08:44.415052 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.405894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.1.32.125:35437)
0620 23:08:44.406939 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.406941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.406948 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:44.406964 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.406965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.406975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.407025 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.407717 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.408356 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.409983 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.410280 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.410322 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.412148 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.412149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.412149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.412156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.413791 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.414207 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.414208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.414209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.414260 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.414487 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.414488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.414489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.414616 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.414724 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.414839 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.414930 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":45,"threads_started":1}
I20260620 23:08:44.415830 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.406262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.1.32.126:35483)
0620 23:08:44.406825 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.406828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.408222 (+ 1394us) server_negotiation.cc:407] Connection header received
0620 23:08:44.408313 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.408313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.408322 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.408350 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.409082 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.409769 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.410923 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.411140 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.411194 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.412959 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.412960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.412961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.412967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.414328 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.414729 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.414730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.414730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.414778 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.415008 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.415009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.415009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.415412 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.415490 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.415661 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.415731 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":93,"threads_started":1}
I20260620 23:08:44.417651 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.405799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.1.32.124:40303)
0620 23:08:44.407532 (+ 1733us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.407535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.407541 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:08:44.407558 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.407558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.407567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.407632 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.410998 (+ 3366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.411619 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.412698 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.413495 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.413566 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.414878 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.414880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.414880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.414887 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.416523 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.416917 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.416918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.416918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.416979 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.417179 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.417180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.417180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.417300 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.417396 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.417500 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.417570 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1720,"thread_start_us":46,"threads_started":1}
I20260620 23:08:44.417281 7237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:44.419162 7237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:44.426453 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.422900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:43369 (local address 127.0.0.1:60888)
0620 23:08:44.423131 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.423140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.423173 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.423468 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.423469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.423471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:44.423519 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.423521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.424364 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.424364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.425313 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.425315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.426203 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.426204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.426261 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.426265 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.426333 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.426382 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":64,"threads_started":1}
I20260620 23:08:44.426455 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.423197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.1.32.67:43369)
0620 23:08:44.423333 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.423337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.423347 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:44.423381 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.423383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.423398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.423469 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:44.423569 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.424282 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.425389 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.426116 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.426186 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.426254 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.426346 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.426381 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260620 23:08:44.426633 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.422791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:35447 (local address 127.0.0.1:49110)
0620 23:08:44.422999 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.423010 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.423051 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.423637 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.423638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.423639 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:44.423662 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.423663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.424792 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.424793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.425677 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.425678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.426407 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.426407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.426476 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.426479 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.426533 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.426574 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260620 23:08:44.426822 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.422790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:37481 (local address 127.0.0.1:60054)
0620 23:08:44.423009 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.423018 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.423057 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.423395 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.423396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.423397 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:44.423447 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.423448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.424672 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.424673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.425700 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.425702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.426597 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.426598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.426657 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.426660 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.426716 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.426752 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":20,"thread_start_us":125,"threads_started":1}
I20260620 23:08:44.426831 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.422949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.1.32.65:37481)
0620 23:08:44.423185 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.423190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.423212 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:08:44.423278 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.423281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.423297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.423396 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:44.423523 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.424582 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.425796 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.426499 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.426596 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.426659 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.426728 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.426762 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
I20260620 23:08:44.427541 8407 tablet_service.cc:1467] Processing CreateTablet for tablet 640df744e0a742d38031485a70a8bc44 (DEFAULT_TABLE table=test-table [id=b5d102e530254d5a8cecf72763379c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:44.427548 8135 tablet_service.cc:1467] Processing CreateTablet for tablet 640df744e0a742d38031485a70a8bc44 (DEFAULT_TABLE table=test-table [id=b5d102e530254d5a8cecf72763379c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:44.427847 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.422938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.1.32.66:35447)
0620 23:08:44.423509 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.423513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.423522 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:44.423557 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.423559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.423574 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.423644 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:44.423720 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.424718 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.425757 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.426319 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.427530 (+ 1211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.427616 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.427642 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.427689 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":87,"threads_started":1}
I20260620 23:08:44.427863 8135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640df744e0a742d38031485a70a8bc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:44.427889 8407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640df744e0a742d38031485a70a8bc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:44.428635 8271 tablet_service.cc:1467] Processing CreateTablet for tablet 640df744e0a742d38031485a70a8bc44 (DEFAULT_TABLE table=test-table [id=b5d102e530254d5a8cecf72763379c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:44.428932 8271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640df744e0a742d38031485a70a8bc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:44.430601 8499 tablet_bootstrap.cc:492] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Bootstrap starting.
I20260620 23:08:44.431173 8499 tablet_bootstrap.cc:654] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:44.431195 8500 tablet_bootstrap.cc:492] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Bootstrap starting.
I20260620 23:08:44.431501 8499 log.cc:824] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:44.431823 8500 tablet_bootstrap.cc:654] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:44.432138 8502 tablet_bootstrap.cc:492] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Bootstrap starting.
I20260620 23:08:44.432142 8500 log.cc:824] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:44.432401 8499 tablet_bootstrap.cc:492] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: No bootstrap required, opened a new log
I20260620 23:08:44.432485 8499 ts_tablet_manager.cc:1397] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:44.432685 8502 tablet_bootstrap.cc:654] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:44.432832 8500 tablet_bootstrap.cc:492] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: No bootstrap required, opened a new log
I20260620 23:08:44.432898 8500 ts_tablet_manager.cc:1397] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:44.432996 8502 log.cc:824] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:44.433735 8502 tablet_bootstrap.cc:492] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: No bootstrap required, opened a new log
I20260620 23:08:44.433799 8502 ts_tablet_manager.cc:1397] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:44.434716 8499 raft_consensus.cc:348] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.434854 8499 raft_consensus.cc:374] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:44.434895 8499 raft_consensus.cc:729] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83163b27b5d94c869492f1391b362a24, State: Initialized, Role: FOLLOWER
I20260620 23:08:44.435051 8499 consensus_queue.cc:260] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.435321 8499 ts_tablet_manager.cc:1428] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260620 23:08:44.435447 8202 heartbeater.cc:503] Master 127.1.32.126:35483 was elected leader, sending a full tablet report...
I20260620 23:08:44.435300 8500 raft_consensus.cc:348] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.435887 8500 raft_consensus.cc:374] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:44.435937 8500 raft_consensus.cc:729] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2ac687bc4843c095f4a4bc474d289b, State: Initialized, Role: FOLLOWER
I20260620 23:08:44.436040 8500 consensus_queue.cc:260] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.436266 8500 ts_tablet_manager.cc:1428] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:44.436277 8502 raft_consensus.cc:348] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.436389 8474 heartbeater.cc:503] Master 127.1.32.126:35483 was elected leader, sending a full tablet report...
I20260620 23:08:44.436409 8502 raft_consensus.cc:374] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:44.436443 8502 raft_consensus.cc:729] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e97115bc9b541b8962c6a974d6f90d6, State: Initialized, Role: FOLLOWER
I20260620 23:08:44.436553 8502 consensus_queue.cc:260] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.436791 8502 ts_tablet_manager.cc:1428] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:44.436959 8338 heartbeater.cc:503] Master 127.1.32.126:35483 was elected leader, sending a full tablet report...
I20260620 23:08:44.521417 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.519154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.0.0.1:49084)
0620 23:08:44.519208 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.519220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.519269 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.519410 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.519411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.519412 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:44.519457 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.519458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.520028 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.520028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.520663 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.520664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.521151 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.521152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.521264 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.521268 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.521307 (+ 39us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.521346 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 23:08:44.521402 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.519214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49084 (local address 127.1.32.124:40303)
0620 23:08:44.519283 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.519285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.519304 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:08:44.519325 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.519325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.519343 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.519406 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:44.519518 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.519975 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.520729 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.521062 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.521150 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.521204 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.521324 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.521353 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260620 23:08:44.521554 8340 tablet.cc:2369] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:44.521765 8030 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260620 23:08:44.522719 8508 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6733ba54101e4f608761e9288c8d5f4c. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:44.522817 8508 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: This master's current role is: LEARNER
W20260620 23:08:44.629292 8204 tablet.cc:2369] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:08:44.647336 8476 tablet.cc:2369] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:44.693696 8507 raft_consensus.cc:482] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:44.693809 8507 raft_consensus.cc:504] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.694108 8507 leader_election.cc:288] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369), 83163b27b5d94c869492f1391b362a24 (127.1.32.65:37481)
I20260620 23:08:44.697927 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.694368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:44337 (local address 127.1.32.65:37481)
0620 23:08:44.694454 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.694456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.694463 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:44.694486 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.694487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.694497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.694567 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:44.694752 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.695464 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.697019 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.697573 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.697633 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.697642 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.697822 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.697860 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:08:44.697927 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.694386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:42015 (local address 127.1.32.67:43369)
0620 23:08:44.694434 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.694436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.694447 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:44.694484 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.694485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.694495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.694567 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:44.694753 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.695448 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.697018 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.697546 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.697625 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.697636 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.697821 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.697860 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 23:08:44.697957 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.694305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:43369 (local address 127.1.32.66:42015)
0620 23:08:44.694363 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.694382 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.694422 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.694584 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.694585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.694587 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:44.694678 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.694680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.695562 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.695563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.696903 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.696907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.697654 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.697656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.697736 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.697741 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.697807 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.697850 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":82}
I20260620 23:08:44.697969 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.694316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:37481 (local address 127.1.32.66:44337)
0620 23:08:44.694363 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:44.694382 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:44.694422 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:44.694584 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:44.694585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:44.694586 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:44.694678 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.694680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.695562 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.695563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.696909 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:44.696910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.697654 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.697656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:44.697740 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.697741 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:08:44.697807 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 23:08:44.697850 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":81}
I20260620 23:08:44.698369 8427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640df744e0a742d38031485a70a8bc44" candidate_uuid: "8e97115bc9b541b8962c6a974d6f90d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2ac687bc4843c095f4a4bc474d289b" is_pre_election: true
I20260620 23:08:44.698434 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640df744e0a742d38031485a70a8bc44" candidate_uuid: "8e97115bc9b541b8962c6a974d6f90d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83163b27b5d94c869492f1391b362a24" is_pre_election: true
I20260620 23:08:44.698539 8427 raft_consensus.cc:2454] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e97115bc9b541b8962c6a974d6f90d6 in term 0.
I20260620 23:08:44.698553 8155 raft_consensus.cc:2454] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e97115bc9b541b8962c6a974d6f90d6 in term 0.
I20260620 23:08:44.698766 8225 leader_election.cc:302] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83163b27b5d94c869492f1391b362a24, 8e97115bc9b541b8962c6a974d6f90d6; no voters:
I20260620 23:08:44.698933 8507 raft_consensus.cc:2781] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:44.698985 8507 raft_consensus.cc:482] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:44.699023 8507 raft_consensus.cc:3037] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:44.699774 8507 raft_consensus.cc:504] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.699936 8507 leader_election.cc:288] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [CANDIDATE]: Term 1 election: Requested vote from peers ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369), 83163b27b5d94c869492f1391b362a24 (127.1.32.65:37481)
I20260620 23:08:44.700088 8427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640df744e0a742d38031485a70a8bc44" candidate_uuid: "8e97115bc9b541b8962c6a974d6f90d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2ac687bc4843c095f4a4bc474d289b"
I20260620 23:08:44.700099 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640df744e0a742d38031485a70a8bc44" candidate_uuid: "8e97115bc9b541b8962c6a974d6f90d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83163b27b5d94c869492f1391b362a24"
I20260620 23:08:44.700167 8427 raft_consensus.cc:3037] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:44.700199 8155 raft_consensus.cc:3037] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:44.700865 8155 raft_consensus.cc:2454] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e97115bc9b541b8962c6a974d6f90d6 in term 1.
I20260620 23:08:44.700870 8427 raft_consensus.cc:2454] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e97115bc9b541b8962c6a974d6f90d6 in term 1.
I20260620 23:08:44.701057 8225 leader_election.cc:302] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83163b27b5d94c869492f1391b362a24, 8e97115bc9b541b8962c6a974d6f90d6; no voters:
I20260620 23:08:44.701175 8507 raft_consensus.cc:2781] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:44.701345 8507 raft_consensus.cc:686] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 LEADER]: Becoming Leader. State: Replica: 8e97115bc9b541b8962c6a974d6f90d6, State: Running, Role: LEADER
I20260620 23:08:44.701488 8507 consensus_queue.cc:237] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } }
I20260620 23:08:44.702472 7236 catalog_manager.cc:5495] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e97115bc9b541b8962c6a974d6f90d6 (127.1.32.66). New cstate: current_term: 1 leader_uuid: "8e97115bc9b541b8962c6a974d6f90d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:44.703233 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: NON_VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: true } } }
I20260620 23:08:44.703358 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:44.704210 8514 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: attempting to promote NON_VOTER f3a030df78544205998dbcb4e5f19c05 to VOTER
I20260620 23:08:44.704346 8514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:44.704821 8030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 LEARNER]: Refusing update from remote peer 6733ba54101e4f608761e9288c8d5f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260620 23:08:44.704869 7844 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Refusing update from remote peer 6733ba54101e4f608761e9288c8d5f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260620 23:08:44.705071 7796 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260620 23:08:44.705188 8514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260620 23:08:44.705845 8514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f3a030df78544205998dbcb4e5f19c05 (127.1.32.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.705991 7844 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f3a030df78544205998dbcb4e5f19c05 (127.1.32.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.706081 8030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f3a030df78544205998dbcb4e5f19c05 (127.1.32.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.706703 8514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.706748 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.706784 8514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:44.706806 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:44.706871 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.706936 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:44.707151 8508 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.707257 8508 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:44.707283 8517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6733ba54101e4f608761e9288c8d5f4c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:44.707388 8517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:44.776485 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.774069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.1.32.66:35447)
0620 23:08:44.774137 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.774139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.774148 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:44.774193 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.774194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.774214 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.774273 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:44.774396 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.774843 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.775746 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.775946 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.776000 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.776142 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:44.776238 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:44.776331 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.776347 (+ 16us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.776373 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:08:44.778314 8427 raft_consensus.cc:1261] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Refusing update from remote peer 8e97115bc9b541b8962c6a974d6f90d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:44.778359 8155 raft_consensus.cc:1261] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 1 FOLLOWER]: Refusing update from remote peer 8e97115bc9b541b8962c6a974d6f90d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:44.778676 8512 consensus_queue.cc:1035] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:44.778811 8507 consensus_queue.cc:1035] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:44.781673 8520 mvcc.cc:205] Tried to move back new op lower bound from 7299059403889414144 to 7299059403577847808. Current Snapshot: MvccSnapshot[applied={T|T < 7299059403889414144}]
I20260620 23:08:44.782153 8521 mvcc.cc:205] Tried to move back new op lower bound from 7299059403889414144 to 7299059403577847808. Current Snapshot: MvccSnapshot[applied={T|T < 7299059403889414144}]
I20260620 23:08:44.783535 7237 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:44.783629 7237 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:44.785218 7237 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6733ba54101e4f608761e9288c8d5f4c: Sending DeleteTablet for 3 replicas of tablet 640df744e0a742d38031485a70a8bc44
I20260620 23:08:44.785487 8271 tablet_service.cc:1514] Processing DeleteTablet for tablet 640df744e0a742d38031485a70a8bc44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:44 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:49110
I20260620 23:08:44.785521 8407 tablet_service.cc:1514] Processing DeleteTablet for tablet 640df744e0a742d38031485a70a8bc44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:44 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:60888
I20260620 23:08:44.785534 8135 tablet_service.cc:1514] Processing DeleteTablet for tablet 640df744e0a742d38031485a70a8bc44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:44 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:60054
I20260620 23:08:44.785687 8528 tablet_replica.cc:330] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: stopping tablet replica
I20260620 23:08:44.785774 8528 raft_consensus.cc:2229] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:44.785861 8528 raft_consensus.cc:2258] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:44.786290 8528 ts_tablet_manager.cc:1905] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:44.786535 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 7205
I20260620 23:08:44.787892 8529 tablet_replica.cc:330] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: stopping tablet replica
I20260620 23:08:44.787990 8529 raft_consensus.cc:2229] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:44.788007 8528 ts_tablet_manager.cc:1918] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:44.788081 8529 raft_consensus.cc:2258] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:44.788175 8528 log.cc:1196] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/640df744e0a742d38031485a70a8bc44
I20260620 23:08:44.788203 8530 tablet_replica.cc:330] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: stopping tablet replica
I20260620 23:08:44.788297 8530 raft_consensus.cc:2229] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:44.788403 8530 raft_consensus.cc:2258] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:44.788671 8528 ts_tablet_manager.cc:1939] T 640df744e0a742d38031485a70a8bc44 P ab2ac687bc4843c095f4a4bc474d289b: Deleting consensus metadata
I20260620 23:08:44.788823 8530 ts_tablet_manager.cc:1905] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:44.790366 8530 ts_tablet_manager.cc:1918] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:44.790449 8530 log.cc:1196] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/640df744e0a742d38031485a70a8bc44
I20260620 23:08:44.790706 8530 ts_tablet_manager.cc:1939] T 640df744e0a742d38031485a70a8bc44 P 83163b27b5d94c869492f1391b362a24: Deleting consensus metadata
W20260620 23:08:44.792634 8360 connection.cc:535] server connection from 127.0.0.1:60888 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 23:08:44.792652 8089 connection.cc:535] server connection from 127.0.0.1:60054 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:08:44.793525 8529 ts_tablet_manager.cc:1905] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:44.795184 8529 ts_tablet_manager.cc:1918] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:44.795259 8529 log.cc:1196] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/640df744e0a742d38031485a70a8bc44
I20260620 23:08:44.795516 8529 ts_tablet_manager.cc:1939] T 640df744e0a742d38031485a70a8bc44 P 8e97115bc9b541b8962c6a974d6f90d6: Deleting consensus metadata
W20260620 23:08:44.795825 8227 connection.cc:441] server connection from 127.0.0.1:49110 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49110 (request call id 1) could send its response
I20260620 23:08:44.801082 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.794269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.1.32.124:40303)
0620 23:08:44.794307 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.794309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.794481 (+ 172us) server_negotiation.cc:407] Connection header received
0620 23:08:44.794529 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.794529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.794539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.794597 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.795316 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.796074 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.796721 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.797045 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.797119 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.798323 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.798324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.798325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.798332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.799988 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.800336 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.800337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.800337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.800394 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.800596 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.800597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.800597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.800721 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.800819 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.800923 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.800992 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:44.805299 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:44.799131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.1.32.125:35437)
0620 23:08:44.799173 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:44.799174 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:44.799288 (+ 114us) server_negotiation.cc:407] Connection header received
0620 23:08:44.799357 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:44.799357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:44.799368 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:44.799433 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:44.799871 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.800556 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.801509 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:44.801695 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:44.801775 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:44.802888 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:44.802889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:44.802889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:44.802896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:44.804199 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.804554 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.804555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.804555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.804605 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:44.804800 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:44.804801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:44.804801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:44.804929 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:44.805024 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:44.805124 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:08:44.805197 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:08:45.782596 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:45.782277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:52615)
0620 23:08:45.782445 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:45.782526 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":66,"threads_started":1}
I20260620 23:08:45.782887 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:45.782430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:40383)
0620 23:08:45.782541 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:45.782590 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":49,"threads_started":1}
W20260620 23:08:45.782943 8474 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:35483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:45.783036 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:45.782772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:46695)
0620 23:08:45.782939 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:45.783000 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":52,"threads_started":1}
W20260620 23:08:45.783089 8202 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:35483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
W20260620 23:08:45.783226 8338 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:35483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:46.285604 8540 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6733ba54101e4f608761e9288c8d5f4c)
I20260620 23:08:46.285714 8541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6733ba54101e4f608761e9288c8d5f4c)
I20260620 23:08:46.285789 8540 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:46.285871 8541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:46.286147 8540 leader_election.cc:288] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483), 28ac6218513c49908d70358f6972b6b3 (127.1.32.125:35437)
I20260620 23:08:46.286346 8541 leader_election.cc:288] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483), f3a030df78544205998dbcb4e5f19c05 (127.1.32.124:40303)
I20260620 23:08:46.286545 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.286241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57286)
0620 23:08:46.286434 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.286497 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":44,"threads_started":1}
W20260620 23:08:46.286672 8006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:46.286814 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.286376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57288)
0620 23:08:46.286717 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.286779 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":66,"threads_started":1}
W20260620 23:08:46.287001 7820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:46.287241 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.287112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57296)
0620 23:08:46.287190 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.287218 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260620 23:08:46.287371 8006 leader_election.cc:334] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483): Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:46.287493 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.287359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57306)
0620 23:08:46.287424 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.287450 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260620 23:08:46.287580 7820 leader_election.cc:334] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483): Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:46.289745 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.286264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:40303 (local address 127.0.0.1:49090)
0620 23:08:46.286540 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.286549 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:46.286594 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:46.286857 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:46.286858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:46.286860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:46.286949 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:46.286952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.287903 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.287904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:46.288657 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:46.288659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.289458 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.289459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:46.289529 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:46.289533 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:46.289598 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:08:46.289644 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":51,"threads_started":1}
I20260620 23:08:46.289758 8545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.286388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.1.32.124:40303)
0620 23:08:46.286658 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:46.286660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:46.286670 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:46.286699 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:46.286700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:46.286711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:46.286797 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:46.287043 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.287791 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.288773 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.289352 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.289437 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:46.289450 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:46.289613 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 23:08:46.289655 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":47,"threads_started":1}
I20260620 23:08:46.289749 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.286456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48480 (local address 127.1.32.125:35437)
0620 23:08:46.287086 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:46.287088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:46.287096 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:46.287116 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:46.287117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:46.287127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:46.287178 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:46.287371 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.287977 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.288843 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.289364 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.289441 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:46.289451 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:46.289613 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 23:08:46.289654 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":45,"threads_started":1}
I20260620 23:08:46.289760 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.286303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:35437 (local address 127.0.0.1:48480)
0620 23:08:46.286493 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.286505 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:46.286564 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:46.287227 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:46.287227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:46.287230 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:46.287282 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:46.287283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.288049 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.288050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:46.288774 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:46.288776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:46.289458 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:46.289459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:46.289535 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:46.289539 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:46.289598 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 23:08:46.289642 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":52,"threads_started":1}
I20260620 23:08:46.289999 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28ac6218513c49908d70358f6972b6b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a030df78544205998dbcb4e5f19c05" is_pre_election: true
I20260620 23:08:46.290130 8030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ac6218513c49908d70358f6972b6b3 in term 1.
I20260620 23:08:46.290248 7844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a030df78544205998dbcb4e5f19c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "28ac6218513c49908d70358f6972b6b3" is_pre_election: true
I20260620 23:08:46.290331 7844 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3a030df78544205998dbcb4e5f19c05 in term 1.
I20260620 23:08:46.290459 7821 leader_election.cc:302] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28ac6218513c49908d70358f6972b6b3, f3a030df78544205998dbcb4e5f19c05; no voters: 6733ba54101e4f608761e9288c8d5f4c
I20260620 23:08:46.290467 8004 leader_election.cc:302] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28ac6218513c49908d70358f6972b6b3, f3a030df78544205998dbcb4e5f19c05; no voters: 6733ba54101e4f608761e9288c8d5f4c
I20260620 23:08:46.290560 8540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 23:08:46.290584 8541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 23:08:46.290602 8540 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6733ba54101e4f608761e9288c8d5f4c)
I20260620 23:08:46.290611 8541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6733ba54101e4f608761e9288c8d5f4c)
I20260620 23:08:46.290640 8541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:08:46.290642 8540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:08:46.291460 8541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:46.291471 8540 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:46.291656 8540 leader_election.cc:288] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [CANDIDATE]: Term 2 election: Requested vote from peers 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483), 28ac6218513c49908d70358f6972b6b3 (127.1.32.125:35437)
I20260620 23:08:46.291719 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28ac6218513c49908d70358f6972b6b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a030df78544205998dbcb4e5f19c05"
I20260620 23:08:46.291800 8030 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 28ac6218513c49908d70358f6972b6b3 in current term 2: Already voted for candidate f3a030df78544205998dbcb4e5f19c05 in this term.
I20260620 23:08:46.291925 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.291807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57330)
0620 23:08:46.291859 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.291890 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:08:46.291656 8541 leader_election.cc:288] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 2 election: Requested vote from peers 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483), f3a030df78544205998dbcb4e5f19c05 (127.1.32.124:40303)
I20260620 23:08:46.291945 7844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a030df78544205998dbcb4e5f19c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "28ac6218513c49908d70358f6972b6b3"
I20260620 23:08:46.292037 7844 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f3a030df78544205998dbcb4e5f19c05 in current term 2: Already voted for candidate 28ac6218513c49908d70358f6972b6b3 in this term.
I20260620 23:08:46.292089 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.291658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57318)
0620 23:08:46.292013 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.292041 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342}
I20260620 23:08:46.292387 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.292284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57332)
0620 23:08:46.292321 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.292357 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260620 23:08:46.292405 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.292323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57346)
0620 23:08:46.292356 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.292379 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260620 23:08:46.292450 7820 leader_election.cc:334] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483): Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
W20260620 23:08:46.292471 8006 leader_election.cc:334] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483): Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:46.292498 7820 leader_election.cc:302] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28ac6218513c49908d70358f6972b6b3; no voters: 6733ba54101e4f608761e9288c8d5f4c, f3a030df78544205998dbcb4e5f19c05
I20260620 23:08:46.292510 8006 leader_election.cc:302] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a030df78544205998dbcb4e5f19c05; no voters: 28ac6218513c49908d70358f6972b6b3, 6733ba54101e4f608761e9288c8d5f4c
I20260620 23:08:46.292567 8540 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 23:08:46.292585 8541 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 23:08:46.783595 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.783307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:42233)
0620 23:08:46.783472 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.783513 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":61,"threads_started":1}
I20260620 23:08:46.783627 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.783385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:39423)
0620 23:08:46.783528 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.783563 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260620 23:08:46.783807 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:46.783558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:33345)
0620 23:08:46.783737 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:46.783769 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":47,"threads_started":1}
I20260620 23:08:47.784430 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:47.784081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:55357)
0620 23:08:47.784315 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:47.784372 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":126,"threads_started":1}
I20260620 23:08:47.784595 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:47.784239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:55941)
0620 23:08:47.784525 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:47.784560 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":65,"threads_started":1}
I20260620 23:08:47.784922 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:47.784469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:35797)
0620 23:08:47.784817 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:47.784873 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":44,"threads_started":1}
I20260620 23:08:47.936757 8556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:47.936856 8556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:47.937100 8556 leader_election.cc:288] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483), f3a030df78544205998dbcb4e5f19c05 (127.1.32.124:40303)
I20260620 23:08:47.937453 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:47.937278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57578)
0620 23:08:47.937373 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:47.937415 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":52,"threads_started":1}
I20260620 23:08:47.937409 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28ac6218513c49908d70358f6972b6b3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a030df78544205998dbcb4e5f19c05" is_pre_election: true
I20260620 23:08:47.937527 8030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28ac6218513c49908d70358f6972b6b3 in term 2.
I20260620 23:08:47.937712 7821 leader_election.cc:302] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ac6218513c49908d70358f6972b6b3, f3a030df78544205998dbcb4e5f19c05; no voters:
I20260620 23:08:47.937820 8556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260620 23:08:47.937860 8556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:47.937882 8556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:08:47.937896 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:47.937774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57582)
0620 23:08:47.937824 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:47.937847 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 23:08:47.938017 7820 leader_election.cc:334] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483): Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:47.954308 8556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:47.954640 8556 leader_election.cc:288] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 3 election: Requested vote from peers 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483), f3a030df78544205998dbcb4e5f19c05 (127.1.32.124:40303)
I20260620 23:08:47.954705 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:47.954572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57592)
0620 23:08:47.954625 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:47.954666 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 23:08:47.954890 8030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28ac6218513c49908d70358f6972b6b3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f3a030df78544205998dbcb4e5f19c05"
I20260620 23:08:47.955005 8030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:08:47.955999 8030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28ac6218513c49908d70358f6972b6b3 in term 3.
I20260620 23:08:47.956220 7821 leader_election.cc:302] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ac6218513c49908d70358f6972b6b3, f3a030df78544205998dbcb4e5f19c05; no voters:
I20260620 23:08:47.956310 8556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 23:08:47.956351 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:47.956265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57594)
0620 23:08:47.956305 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:47.956328 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 23:08:47.956420 7820 leader_election.cc:334] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483): Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
I20260620 23:08:47.956457 8556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [term 3 LEADER]: Becoming Leader. State: Replica: 28ac6218513c49908d70358f6972b6b3, State: Running, Role: LEADER
I20260620 23:08:47.956521 8556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } }
I20260620 23:08:47.956902 8559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28ac6218513c49908d70358f6972b6b3. Latest consensus state: current_term: 3 leader_uuid: "28ac6218513c49908d70358f6972b6b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:47.957001 8559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:47.957181 8561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:47.957633 8561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:47.957706 8561 catalog_manager.cc:1259] Loaded cluster ID: 3d778f7abbef4e43944afd7af343d278
I20260620 23:08:47.957741 8561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:47.957918 8561 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:47.958009 8561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Loaded TSK: 0
I20260620 23:08:47.958153 8561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:08:48.040232 7834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:48.042454 7834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:48.043944 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.043804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57602)
0620 23:08:48.043867 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.043918 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 23:08:48.043951 8030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [term 3 FOLLOWER]: Refusing update from remote peer 28ac6218513c49908d70358f6972b6b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260620 23:08:48.044261 8559 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260620 23:08:48.044327 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.044214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.0.0.1:57612)
0620 23:08:48.044271 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.044303 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 23:08:48.044438 7820 consensus_peers.cc:487] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 -> Peer 6733ba54101e4f608761e9288c8d5f4c (127.1.32.126:35483): Couldn't send request to peer 6733ba54101e4f608761e9288c8d5f4c. Status: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 23:08:48.044886 8563 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28ac6218513c49908d70358f6972b6b3. Latest consensus state: current_term: 3 leader_uuid: "28ac6218513c49908d70358f6972b6b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:48.044997 8563 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:48.045384 8562 mvcc.cc:205] Tried to move back new op lower bound from 7299059417265721344 to 7299059416910737408. Current Snapshot: MvccSnapshot[applied={T|T < 7299059417265721344}]
I20260620 23:08:48.045567 8564 mvcc.cc:205] Tried to move back new op lower bound from 7299059417265721344 to 7299059416910737408. Current Snapshot: MvccSnapshot[applied={T|T < 7299059417265721344}]
I20260620 23:08:48.045538 8556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "28ac6218513c49908d70358f6972b6b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:48.045625 8556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:48.045691 8563 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "28ac6218513c49908d70358f6972b6b3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "6733ba54101e4f608761e9288c8d5f4c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 35483 } } peers { permanent_uuid: "28ac6218513c49908d70358f6972b6b3" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 35437 } attrs { promote: false } } peers { permanent_uuid: "f3a030df78544205998dbcb4e5f19c05" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 40303 } attrs { promote: false } } }
I20260620 23:08:48.046216 8563 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a030df78544205998dbcb4e5f19c05 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:08:48.050724 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.047385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:37481 (local address 127.0.0.1:60070)
0620 23:08:48.047436 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.047448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:48.047501 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:48.047906 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:48.047907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:48.047909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:48.047964 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.047966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.048878 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.048879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.049650 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.049652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.050451 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.050452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.050509 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.050514 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:08:48.050590 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:08:48.050639 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
I20260620 23:08:48.050832 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.047461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:43369 (local address 127.0.0.1:60892)
0620 23:08:48.047560 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.047570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:48.047614 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:48.048042 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:48.048043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:48.048044 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:48.048078 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.048078 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.048990 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.048991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.049753 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.049756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.050537 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.050538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.050623 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.050627 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:48.050718 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 23:08:48.050760 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":62,"threads_started":1}
I20260620 23:08:48.050999 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.047611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.1.32.67:43369)
0620 23:08:48.047783 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:48.047786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:48.047794 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:48.047822 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:48.047823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:48.047848 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:48.047923 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:48.048142 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.048853 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.049856 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.050432 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.050530 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.050544 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:48.050726 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 23:08:48.050770 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":50,"threads_started":1}
I20260620 23:08:48.051003 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.047464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.1.32.65:37481)
0620 23:08:48.047744 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:48.047746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:48.047756 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:48.047792 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:48.047793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:48.047807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:48.047901 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:48.048023 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.048789 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.049759 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.050353 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.050446 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.050462 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:48.050852 (+ 390us) server_negotiation.cc:299] Negotiation successful
0620 23:08:48.050938 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":65,"threads_started":1}
I20260620 23:08:48.051235 8407 tablet_service.cc:1467] Processing CreateTablet for tablet afd2dd7f4d604404af0218b6780a4b55 (DEFAULT_TABLE table=test-table [id=c50b9462a77846c08312f9a852a44ffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:48.051363 8407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd2dd7f4d604404af0218b6780a4b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:48.051383 8135 tablet_service.cc:1467] Processing CreateTablet for tablet afd2dd7f4d604404af0218b6780a4b55 (DEFAULT_TABLE table=test-table [id=c50b9462a77846c08312f9a852a44ffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:48.051501 8135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd2dd7f4d604404af0218b6780a4b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:48.051982 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.047615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:35447 (local address 127.0.0.1:49124)
0620 23:08:48.047724 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.047733 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:48.047775 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:48.048168 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:48.048169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:48.048170 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:48.048204 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.048279 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.049745 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.049746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.050638 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.050639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.051729 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.051730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.051799 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.051804 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:08:48.051875 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 23:08:48.051913 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260620 23:08:48.052340 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.047847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.1.32.66:35447)
0620 23:08:48.048051 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:48.048053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:48.048060 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:48.048086 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:48.048087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:48.048097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:48.048168 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:48.048382 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.048927 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.051099 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.051655 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.051728 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.051752 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:48.051894 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 23:08:48.051944 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":48,"threads_started":1}
I20260620 23:08:48.052515 8271 tablet_service.cc:1467] Processing CreateTablet for tablet afd2dd7f4d604404af0218b6780a4b55 (DEFAULT_TABLE table=test-table [id=c50b9462a77846c08312f9a852a44ffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:48.052605 8271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd2dd7f4d604404af0218b6780a4b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:48.052858 8574 tablet_bootstrap.cc:492] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24: Bootstrap starting.
I20260620 23:08:48.053210 8575 tablet_bootstrap.cc:492] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: Bootstrap starting.
I20260620 23:08:48.053264 8574 tablet_bootstrap.cc:654] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:48.053582 8575 tablet_bootstrap.cc:654] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:48.054116 8578 tablet_bootstrap.cc:492] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6: Bootstrap starting.
I20260620 23:08:48.054203 8574 tablet_bootstrap.cc:492] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24: No bootstrap required, opened a new log
I20260620 23:08:48.054296 8574 ts_tablet_manager.cc:1397] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:48.054486 8574 raft_consensus.cc:348] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.054570 8574 raft_consensus.cc:374] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:48.054595 8578 tablet_bootstrap.cc:654] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:48.054623 8574 raft_consensus.cc:729] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83163b27b5d94c869492f1391b362a24, State: Initialized, Role: FOLLOWER
I20260620 23:08:48.054692 8574 consensus_queue.cc:260] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.054958 8574 ts_tablet_manager.cc:1428] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:08:48.055346 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.055214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:48939)
0620 23:08:48.055260 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.055292 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 23:08:48.055660 8578 tablet_bootstrap.cc:492] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6: No bootstrap required, opened a new log
I20260620 23:08:48.055751 8578 ts_tablet_manager.cc:1397] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:48.055912 8575 tablet_bootstrap.cc:492] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: No bootstrap required, opened a new log
I20260620 23:08:48.055909 8578 raft_consensus.cc:348] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.055987 8578 raft_consensus.cc:374] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:48.055987 8575 ts_tablet_manager.cc:1397] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 23:08:48.056160 8575 raft_consensus.cc:348] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.056231 8575 raft_consensus.cc:374] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:48.056254 8575 raft_consensus.cc:729] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2ac687bc4843c095f4a4bc474d289b, State: Initialized, Role: FOLLOWER
I20260620 23:08:48.056330 8575 consensus_queue.cc:260] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.056478 8578 raft_consensus.cc:729] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e97115bc9b541b8962c6a974d6f90d6, State: Initialized, Role: FOLLOWER
I20260620 23:08:48.056533 8575 ts_tablet_manager.cc:1428] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:48.056581 8578 consensus_queue.cc:260] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.056790 8578 ts_tablet_manager.cc:1428] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:08:48.057163 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.057029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:53653)
0620 23:08:48.057097 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.057135 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 23:08:48.057479 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.057388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:51369)
0620 23:08:48.057427 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.057453 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260620 23:08:48.057857 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.057740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:37409)
0620 23:08:48.057808 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.057833 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 23:08:48.057905 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.057783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:35621)
0620 23:08:48.057842 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.057873 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 23:08:48.059087 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.058975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:50923)
0620 23:08:48.059029 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.059056 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:08:48.059247 8203 heartbeater.cc:503] Master 127.1.32.125:35437 was elected leader, sending a full tablet report...
I20260620 23:08:48.059257 8475 heartbeater.cc:503] Master 127.1.32.125:35437 was elected leader, sending a full tablet report...
I20260620 23:08:48.108446 8580 raft_consensus.cc:482] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:48.108551 8580 raft_consensus.cc:504] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.108902 8580 leader_election.cc:288] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369), 8e97115bc9b541b8962c6a974d6f90d6 (127.1.32.66:35447)
I20260620 23:08:48.112381 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.109320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:40871 (local address 127.1.32.66:35447)
0620 23:08:48.109366 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:48.109368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:48.109456 (+ 88us) server_negotiation.cc:407] Connection header received
0620 23:08:48.109519 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:48.109520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:48.109531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:48.109604 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:48.109736 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.110184 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.111393 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.112011 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.112106 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.112118 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:48.112270 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:08:48.112307 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:08:48.112416 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.109170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:54511 (local address 127.1.32.67:43369)
0620 23:08:48.109232 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:48.109235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:48.109246 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:48.109273 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:48.109274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:48.109287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:48.109374 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:48.109561 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.110022 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.111378 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.112019 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.112113 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.112124 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:48.112305 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 23:08:48.112339 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:08:48.112406 8583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.109214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:35447 (local address 127.1.32.65:40871)
0620 23:08:48.109390 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.109398 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:48.109468 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:48.109610 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:48.109610 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:48.109612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:48.109670 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.109672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.110261 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.110262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.111282 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.111285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.112109 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.112110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.112192 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.112196 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:48.112261 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:08:48.112305 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260620 23:08:48.112423 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.109080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:43369 (local address 127.1.32.65:54511)
0620 23:08:48.109128 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.109140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:48.109194 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:48.109382 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:48.109384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:48.109387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:48.109483 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.109485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.110104 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.110105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.111272 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:48.111274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.112182 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.112183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:48.112235 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.112238 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:48.112299 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:08:48.112335 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":2}
I20260620 23:08:48.112841 8427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2dd7f4d604404af0218b6780a4b55" candidate_uuid: "83163b27b5d94c869492f1391b362a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2ac687bc4843c095f4a4bc474d289b" is_pre_election: true
I20260620 23:08:48.112882 8291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2dd7f4d604404af0218b6780a4b55" candidate_uuid: "83163b27b5d94c869492f1391b362a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e97115bc9b541b8962c6a974d6f90d6" is_pre_election: true
I20260620 23:08:48.112951 8427 raft_consensus.cc:2454] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83163b27b5d94c869492f1391b362a24 in term 0.
I20260620 23:08:48.113027 8291 raft_consensus.cc:2454] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83163b27b5d94c869492f1391b362a24 in term 0.
I20260620 23:08:48.113193 8090 leader_election.cc:302] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83163b27b5d94c869492f1391b362a24, ab2ac687bc4843c095f4a4bc474d289b; no voters:
I20260620 23:08:48.113307 8580 raft_consensus.cc:2781] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:48.113343 8580 raft_consensus.cc:482] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:48.113370 8580 raft_consensus.cc:3037] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:48.154034 8580 raft_consensus.cc:504] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.154263 8580 leader_election.cc:288] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [CANDIDATE]: Term 1 election: Requested vote from peers ab2ac687bc4843c095f4a4bc474d289b (127.1.32.67:43369), 8e97115bc9b541b8962c6a974d6f90d6 (127.1.32.66:35447)
I20260620 23:08:48.154518 8427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2dd7f4d604404af0218b6780a4b55" candidate_uuid: "83163b27b5d94c869492f1391b362a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2ac687bc4843c095f4a4bc474d289b"
I20260620 23:08:48.154515 8291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2dd7f4d604404af0218b6780a4b55" candidate_uuid: "83163b27b5d94c869492f1391b362a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e97115bc9b541b8962c6a974d6f90d6"
I20260620 23:08:48.154614 8427 raft_consensus.cc:3037] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:48.154631 8291 raft_consensus.cc:3037] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:48.155522 8291 raft_consensus.cc:2454] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83163b27b5d94c869492f1391b362a24 in term 1.
I20260620 23:08:48.155534 8427 raft_consensus.cc:2454] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83163b27b5d94c869492f1391b362a24 in term 1.
I20260620 23:08:48.155719 8090 leader_election.cc:302] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83163b27b5d94c869492f1391b362a24, 8e97115bc9b541b8962c6a974d6f90d6; no voters:
I20260620 23:08:48.155841 8580 raft_consensus.cc:2781] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:48.156032 8580 raft_consensus.cc:686] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 1 LEADER]: Becoming Leader. State: Replica: 83163b27b5d94c869492f1391b362a24, State: Running, Role: LEADER
I20260620 23:08:48.156116 8580 consensus_queue.cc:237] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } }
I20260620 23:08:48.156481 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.156350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:55621)
0620 23:08:48.156404 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.156445 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 23:08:48.156924 7832 catalog_manager.cc:5495] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83163b27b5d94c869492f1391b362a24 (127.1.32.65). New cstate: current_term: 1 leader_uuid: "83163b27b5d94c869492f1391b362a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83163b27b5d94c869492f1391b362a24" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 37481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:48.170218 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.168105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60074 (local address 127.1.32.65:37481)
0620 23:08:48.168171 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:48.168172 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:48.168181 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:48.168228 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:48.168229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:48.168247 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:48.168301 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:48.168444 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.168910 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.169616 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:48.169796 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:48.169848 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:48.169959 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:48.170063 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:48.170116 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:48.170151 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 23:08:48.170167 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:08:48.171335 8427 raft_consensus.cc:1261] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Refusing update from remote peer 83163b27b5d94c869492f1391b362a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:48.171456 8291 raft_consensus.cc:1261] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 FOLLOWER]: Refusing update from remote peer 83163b27b5d94c869492f1391b362a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:48.171758 8580 consensus_queue.cc:1035] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab2ac687bc4843c095f4a4bc474d289b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:48.171900 8584 consensus_queue.cc:1035] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e97115bc9b541b8962c6a974d6f90d6" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:48.172530 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.172409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:48677)
0620 23:08:48.172462 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.172494 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 23:08:48.172981 8339 heartbeater.cc:503] Master 127.1.32.125:35437 was elected leader, sending a full tablet report...
I20260620 23:08:48.173375 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.173189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:52269)
0620 23:08:48.173318 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.173348 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260620 23:08:48.174258 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.174167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:34633)
0620 23:08:48.174200 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.174227 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260620 23:08:48.174916 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.174807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.66:44395)
0620 23:08:48.174847 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.174870 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 23:08:48.175581 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.175458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.65:59141)
0620 23:08:48.175494 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.175527 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 23:08:48.175560 7832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:48.175642 7832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:48.176589 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.176492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:35483 (local address 127.1.32.67:56197)
0620 23:08:48.176534 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.176558 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 23:08:48.177273 7832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28ac6218513c49908d70358f6972b6b3: Sending DeleteTablet for 3 replicas of tablet afd2dd7f4d604404af0218b6780a4b55
I20260620 23:08:48.177477 8407 tablet_service.cc:1514] Processing DeleteTablet for tablet afd2dd7f4d604404af0218b6780a4b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:48 UTC) from {username='oryx', principal='oryx/127.1.32.125@KRBTEST.COM'} at 127.0.0.1:60892
I20260620 23:08:48.177546 8135 tablet_service.cc:1514] Processing DeleteTablet for tablet afd2dd7f4d604404af0218b6780a4b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:48 UTC) from {username='oryx', principal='oryx/127.1.32.125@KRBTEST.COM'} at 127.0.0.1:60070
I20260620 23:08:48.177567 8271 tablet_service.cc:1514] Processing DeleteTablet for tablet afd2dd7f4d604404af0218b6780a4b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:48 UTC) from {username='oryx', principal='oryx/127.1.32.125@KRBTEST.COM'} at 127.0.0.1:49124
I20260620 23:08:48.177686 8406 tablet_service.cc:1514] Processing DeleteTablet for tablet afd2dd7f4d604404af0218b6780a4b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:48 UTC) from {username='oryx', principal='oryx/127.1.32.125@KRBTEST.COM'} at 127.0.0.1:60892
I20260620 23:08:48.177805 8599 tablet_replica.cc:330] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24: stopping tablet replica
I20260620 23:08:48.177857 8599 raft_consensus.cc:2229] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:48.177867 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 8072
I20260620 23:08:48.177933 8599 raft_consensus.cc:2258] T afd2dd7f4d604404af0218b6780a4b55 P 83163b27b5d94c869492f1391b362a24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:48.178473 8598 tablet_replica.cc:330] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: stopping tablet replica
I20260620 23:08:48.178525 8598 raft_consensus.cc:2229] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:48.178562 8598 raft_consensus.cc:2258] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:48.178699 8600 tablet_replica.cc:330] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6: stopping tablet replica
I20260620 23:08:48.178752 8600 raft_consensus.cc:2229] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:48.178791 8600 raft_consensus.cc:2258] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:48.179203 8600 ts_tablet_manager.cc:1905] T afd2dd7f4d604404af0218b6780a4b55 P 8e97115bc9b541b8962c6a974d6f90d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:48.179263 8598 ts_tablet_manager.cc:1905] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:48.183956 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.183866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:37481 (local address 127.0.0.1:60080)
0620 23:08:48.183904 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.183927 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260620 23:08:48.184105 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 8208
W20260620 23:08:48.184128 7819 catalog_manager.cc:4584] TS 83163b27b5d94c869492f1391b362a24 (127.1.32.65:37481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet afd2dd7f4d604404af0218b6780a4b55: Network error: Client connection negotiation failed: client connection to 127.1.32.65:37481: connect: Connection refused (error 111)
I20260620 23:08:48.236302 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.236137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:37481 (local address 127.0.0.1:60082)
0620 23:08:48.236228 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.236264 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260620 23:08:48.236704 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.236611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:37481 (local address 127.0.0.1:60098)
0620 23:08:48.236657 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.236685 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 23:08:48.258988 8598 ts_tablet_manager.cc:1918] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:48.259087 8598 log.cc:1196] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/afd2dd7f4d604404af0218b6780a4b55
I20260620 23:08:48.259362 8598 ts_tablet_manager.cc:1939] T afd2dd7f4d604404af0218b6780a4b55 P ab2ac687bc4843c095f4a4bc474d289b: Deleting consensus metadata
I20260620 23:08:48.261487 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 8344
I20260620 23:08:48.261580 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.261434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:35447 (local address 127.0.0.1:49128)
0620 23:08:48.261495 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.261525 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:35447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 23:08:48.278185 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 7701
I20260620 23:08:48.278198 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:48.278084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:43369 (local address 127.0.0.1:60908)
0620 23:08:48.278142 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:48.278163 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.67:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 23:08:48.282981 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 7888
2026-06-20T23:08:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6539 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:48 dist-test-slave-fzjm krb5kdc[8603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:48 dist-test-slave-fzjm krb5kdc[8603](info): set up 2 sockets
Jun 20 23:08:48 dist-test-slave-fzjm krb5kdc[8603](info): commencing operation
krb5kdc: starting...
W20260620 23:08:49.419140 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:49 dist-test-slave-fzjm krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab.
Entry for principal oryx/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab.
I20260620 23:08:49.525530 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39667
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab
--principal=oryx/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:39667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:49.601136 8616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:49.601298 8616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:49.601317 8616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:49.602319 8616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:49.602355 8616 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 23:08:49.602366 8616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:49.602377 8616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:49.602388 8616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:49.602430 8616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:49.603763 8616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:39667
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.126.keytab
--principal=oryx/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:49.603988 8616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:49.604205 8616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:49.608793 8622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:49.608798 8621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:49.608803 8624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:49.608819 8616 server_base.cc:1034] running on GCE node
I20260620 23:08:49.609201 8616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:49.609432 8616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:49.610572 8616 hybrid_clock.cc:648] HybridClock initialized: now 1781996929610557 us; error 29 us; skew 500 ppm
Jun 20 23:08:49 dist-test-slave-fzjm krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:49.612959 8616 init.cc:377] Logged in from keytab as oryx/127.1.32.126@KRBTEST.COM (short username oryx)
I20260620 23:08:49.613317 8616 webserver.cc:460] Webserver started at http://127.1.32.126:42901/ using document root <none> and password file <none>
I20260620 23:08:49.613507 8616 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:49.613569 8616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:49.613657 8616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:49.614673 8616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "1513ac8b1b674e5ca355233fb9f0886e"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "f219c984927bab2ae8a2b14504f3280c"
server_key_iv: "44190b9ea7d37dc3fa2059ccfa657f5d"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.615051 8616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "1513ac8b1b674e5ca355233fb9f0886e"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "f219c984927bab2ae8a2b14504f3280c"
server_key_iv: "44190b9ea7d37dc3fa2059ccfa657f5d"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.616313 8616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:49.616958 8631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:49.617096 8616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:49.617199 8616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "1513ac8b1b674e5ca355233fb9f0886e"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "f219c984927bab2ae8a2b14504f3280c"
server_key_iv: "44190b9ea7d37dc3fa2059ccfa657f5d"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.617249 8616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:49.646116 8616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:49.647976 8616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:49.648077 8616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:49.652704 8616 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:39667
I20260620 23:08:49.652768 8693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:39667 every 8 connection(s)
I20260620 23:08:49.653062 8616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:49.653684 8694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:49.655740 8694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Bootstrap starting.
I20260620 23:08:49.656241 8694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:49.656466 8694 log.cc:824] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:49.657051 8694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: No bootstrap required, opened a new log
I20260620 23:08:49.658918 8694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1513ac8b1b674e5ca355233fb9f0886e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39667 } }
I20260620 23:08:49.659054 8694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:49.659075 8694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1513ac8b1b674e5ca355233fb9f0886e, State: Initialized, Role: FOLLOWER
I20260620 23:08:49.659195 8694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1513ac8b1b674e5ca355233fb9f0886e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39667 } }
I20260620 23:08:49.659265 8694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:49.659286 8694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:49.659317 8694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:49.659498 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 8616
I20260620 23:08:49.659567 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:49.659754 1153 external_mini_cluster.cc:1428] Setting key d833e3aeb8518100c2889b6f2ed90226
I20260620 23:08:49.659832 8694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1513ac8b1b674e5ca355233fb9f0886e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39667 } }
I20260620 23:08:49.659966 8694 leader_election.cc:302] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1513ac8b1b674e5ca355233fb9f0886e; no voters:
I20260620 23:08:49.660122 8694 leader_election.cc:288] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:49.660271 8697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:49.660355 8694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:49.660471 8697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [term 1 LEADER]: Becoming Leader. State: Replica: 1513ac8b1b674e5ca355233fb9f0886e, State: Running, Role: LEADER
I20260620 23:08:49.660602 8697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1513ac8b1b674e5ca355233fb9f0886e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39667 } }
I20260620 23:08:49.661064 8699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1513ac8b1b674e5ca355233fb9f0886e. Latest consensus state: current_term: 1 leader_uuid: "1513ac8b1b674e5ca355233fb9f0886e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1513ac8b1b674e5ca355233fb9f0886e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39667 } } }
I20260620 23:08:49.661312 8699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:49.661716 8708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:49.662516 8708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:49.662541 8701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1513ac8b1b674e5ca355233fb9f0886e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1513ac8b1b674e5ca355233fb9f0886e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39667 } } }
I20260620 23:08:49.662801 8701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:49.664652 8708 catalog_manager.cc:1347] Generated new cluster ID: 988c80de4a264ac19e527150d7887a9d
I20260620 23:08:49.664705 8708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:08:49 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:49.669135 8708 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:49.669754 8708 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:49.670449 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.660313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53320 (local address 127.1.32.126:39667)
0620 23:08:49.660538 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:49.660546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:49.660572 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:08:49.660676 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:49.660681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:49.660731 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:49.661795 (+ 1064us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:49.661923 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.663383 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.664339 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.664875 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.665003 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.667279 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:49.667282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:49.667283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:49.667294 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:49.668933 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.669281 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.669284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.669284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.669380 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.669562 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.669563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.669563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.669790 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:49.669940 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:49.670143 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 23:08:49.670305 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260620 23:08:49.674607 8708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Generated new TSK 0
I20260620 23:08:49.674736 8708 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab.
Entry for principal oryx/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab.
I20260620 23:08:49.699585 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39667
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab
--principal=oryx/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:49.784606 8720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:49.784803 8720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:49.784827 8720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:49.785938 8720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:49.785974 8720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:49.785995 8720 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:49.786011 8720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:49.787518 8720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab
--principal=oryx/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:49.787730 8720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:49.787966 8720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:49.793356 8726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:49.793465 8728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:49.793356 8725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:49.793629 8720 server_base.cc:1034] running on GCE node
I20260620 23:08:49.793959 8720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:49.794195 8720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:49.795393 8720 hybrid_clock.cc:648] HybridClock initialized: now 1781996929795376 us; error 30 us; skew 500 ppm
Jun 20 23:08:49 dist-test-slave-fzjm krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:49.798077 8720 init.cc:377] Logged in from keytab as oryx/127.1.32.65@KRBTEST.COM (short username oryx)
I20260620 23:08:49.798524 8720 webserver.cc:460] Webserver started at http://127.1.32.65:33051/ using document root <none> and password file <none>
I20260620 23:08:49.798784 8720 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:49.798848 8720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:49.799016 8720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:49.800017 8720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "3f0a5708a012449ab80fa30286c7f82e"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "a6a9262f53f558f8c53144952522fbe2"
server_key_iv: "8b5ba2a948d85c12c19cbf6565916843"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.800345 8720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "3f0a5708a012449ab80fa30286c7f82e"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "a6a9262f53f558f8c53144952522fbe2"
server_key_iv: "8b5ba2a948d85c12c19cbf6565916843"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.801645 8720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:49.802337 8735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:49.802487 8720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:49.802582 8720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "3f0a5708a012449ab80fa30286c7f82e"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "a6a9262f53f558f8c53144952522fbe2"
server_key_iv: "8b5ba2a948d85c12c19cbf6565916843"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.802647 8720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:49.812948 8720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:49.815243 8720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:49.815392 8720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:49.815887 8720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:49.815943 8720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:49.815981 8720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:49.816027 8720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:08:49 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:49.823355 8720 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:38029
I20260620 23:08:49.823424 8850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:38029 every 8 connection(s)
I20260620 23:08:49.823741 8720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:49.824052 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 8720
I20260620 23:08:49.824172 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:49.824379 1153 external_mini_cluster.cc:1428] Setting key 8c830c0579df72d2ef1b6ebf0f08d1c8
I20260620 23:08:49.826417 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.816447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:58711 (local address 127.1.32.126:39667)
0620 23:08:49.816497 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:49.816499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:49.816706 (+ 207us) server_negotiation.cc:407] Connection header received
0620 23:08:49.817297 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:49.817298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:49.817315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:49.817377 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:49.818222 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.819078 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.820220 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.820547 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.820609 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.823453 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:49.823454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:49.823455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:49.823462 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:49.824999 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.825447 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.825447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.825448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.825497 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.825755 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.825755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.825755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.825861 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:49.825913 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:49.826207 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 23:08:49.826298 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 23:08:49.826597 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.816446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39667 (local address 127.1.32.65:58711)
0620 23:08:49.816598 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:49.816630 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:49.817161 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:49.817478 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:49.817482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:49.817838 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:49.818121 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.818127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.819193 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.819194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.820113 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.820116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.820684 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.820685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.820854 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.821423 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:49.821435 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:49.823350 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:49.825062 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.825068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.825070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.825373 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.825559 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.825560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.825560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.825697 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.825920 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:49.825925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:49.826081 (+ 156us) client_negotiation.cc:769] Sending connection context
0620 23:08:49.826189 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 23:08:49.826357 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260620 23:08:49.827169 8851 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39667
I20260620 23:08:49.827255 8851 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:49.827445 8851 heartbeater.cc:511] Master 127.1.32.126:39667 requested a full tablet report, sending...
I20260620 23:08:49.827957 8647 ts_manager.cc:194] Registered new tserver with Master: 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029)
I20260620 23:08:49.828650 8647 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:58711
WARNING: no policy specified for oryx/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:08:49.834439 8858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.828160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39667 (local address 127.1.32.65:38095)
0620 23:08:49.828911 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:49.828922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:49.829358 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:49.829502 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:49.829502 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:49.829665 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:49.829715 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.829716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.830284 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.830285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.830736 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.830737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.831163 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.831164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.831233 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.831666 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:49.831670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:49.831930 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:49.833477 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.833478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.833479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.833641 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.833858 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.833859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.833859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.833916 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.834135 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:49.834136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:49.834179 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 23:08:49.834265 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:08:49.834353 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":79,"threads_started":1}
I20260620 23:08:49.834445 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.828207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:38095 (local address 127.1.32.126:39667)
0620 23:08:49.828238 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:49.828239 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:49.828985 (+ 746us) server_negotiation.cc:407] Connection header received
0620 23:08:49.829445 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:49.829446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:49.829455 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:49.829499 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:49.829794 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.830202 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.830808 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.831054 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.831160 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.832014 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:49.832015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:49.832016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:49.832022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:49.833397 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.833721 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.833721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.833722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.833774 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.833974 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.833975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.833975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.834075 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:49.834142 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:49.834286 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 23:08:49.834375 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab.
Entry for principal oryx/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab.
I20260620 23:08:49.847981 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39667
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab
--principal=oryx/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:49.922842 8860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:49.923067 8860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:49.923087 8860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:49.924127 8860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:49.924166 8860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:49.924186 8860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:49.924202 8860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:49.925588 8860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab
--principal=oryx/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:49.925777 8860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:49.925943 8860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:49.930789 8866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:49.930804 8865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:49.930781 8868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:49.930835 8860 server_base.cc:1034] running on GCE node
I20260620 23:08:49.931201 8860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:49.931433 8860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:49.932588 8860 hybrid_clock.cc:648] HybridClock initialized: now 1781996929932573 us; error 27 us; skew 500 ppm
Jun 20 23:08:49 dist-test-slave-fzjm krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:49.935111 8860 init.cc:377] Logged in from keytab as oryx/127.1.32.66@KRBTEST.COM (short username oryx)
I20260620 23:08:49.935515 8860 webserver.cc:460] Webserver started at http://127.1.32.66:41345/ using document root <none> and password file <none>
I20260620 23:08:49.935750 8860 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:49.935798 8860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:49.935974 8860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:49.936990 8860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "1afb552a650f457c99fac04f93ae69b0"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "00ffdee76909f874bbf06a9a833bae2c"
server_key_iv: "0ceefb84e61a01f51b850bc3dc65c0af"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.937348 8860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "1afb552a650f457c99fac04f93ae69b0"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "00ffdee76909f874bbf06a9a833bae2c"
server_key_iv: "0ceefb84e61a01f51b850bc3dc65c0af"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.938989 8860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260620 23:08:49.939752 8875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:49.939934 8860 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:49.940021 8860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "1afb552a650f457c99fac04f93ae69b0"
format_stamp: "Formatted at 2026-06-20 23:08:49 on dist-test-slave-fzjm"
server_key: "00ffdee76909f874bbf06a9a833bae2c"
server_key_iv: "0ceefb84e61a01f51b850bc3dc65c0af"
server_key_version: "encryptionkey@0"
I20260620 23:08:49.940073 8860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:49.955536 8860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:49.957557 8860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:49.957680 8860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:49.958115 8860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:49.958153 8860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:49.958174 8860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:49.958216 8860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:49.965402 8860 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:36713
I20260620 23:08:49.965461 8990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:36713 every 8 connection(s)
Jun 20 23:08:49 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:49.965756 8860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:49.968649 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.958591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:47703 (local address 127.1.32.126:39667)
0620 23:08:49.958636 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:49.958638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:49.958903 (+ 265us) server_negotiation.cc:407] Connection header received
0620 23:08:49.959421 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:49.959423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:49.959435 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:49.959495 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:49.960416 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.961313 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.963117 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.963457 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.963527 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.966257 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:49.966258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:49.966259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:49.966267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:49.967539 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.967894 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.967895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.967896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.967943 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.968149 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.968149 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.968150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.968264 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:49.968310 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:49.968514 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 23:08:49.968583 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:49.968887 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.958585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39667 (local address 127.1.32.66:47703)
0620 23:08:49.958754 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:49.958800 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:49.959304 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:49.959608 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:49.959613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:49.959991 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:49.960293 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.960298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.961406 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.961407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.963028 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.963032 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.963537 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.963538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.963692 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.964333 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:49.964344 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:49.966143 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:49.967607 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.967613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.967615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.967831 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.967994 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.967995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.967995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.968082 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.968313 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:49.968318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:49.968427 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 23:08:49.968501 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:08:49.968618 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":111,"threads_started":1}
I20260620 23:08:49.969323 8991 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39667
I20260620 23:08:49.969445 8991 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:49.969638 8991 heartbeater.cc:511] Master 127.1.32.126:39667 requested a full tablet report, sending...
I20260620 23:08:49.970041 8648 ts_manager.cc:194] Registered new tserver with Master: 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713)
I20260620 23:08:49.970466 8648 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:47703
I20260620 23:08:49.971912 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 8860
I20260620 23:08:49.971985 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:49.972121 1153 external_mini_cluster.cc:1428] Setting key 2ad5f4cd4323d25e91da40b0a9118406
I20260620 23:08:49.976491 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.970401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39667 (local address 127.1.32.66:36253)
0620 23:08:49.970563 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:49.970585 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:49.970653 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:49.970792 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:49.970792 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:49.971047 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:49.971099 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.971101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.971767 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.971768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.972489 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:49.972491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.972878 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.972879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:49.972941 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.973356 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:49.973359 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:49.973663 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:49.975471 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.975472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.975472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.975629 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.975839 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:49.975839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:49.975840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:49.975890 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:49.976219 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:49.976220 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:49.976251 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 23:08:49.976300 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:08:49.976430 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260620 23:08:49.976567 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:49.970484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:36253 (local address 127.1.32.126:39667)
0620 23:08:49.970536 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:49.970538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:49.970641 (+ 103us) server_negotiation.cc:407] Connection header received
0620 23:08:49.970715 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:49.970716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:49.970727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:49.970786 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:49.971181 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.971674 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.972582 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:49.972785 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:49.972866 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:49.973768 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:49.973769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:49.973769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:49.973776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:49.975372 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.975706 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.975707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.975708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.975772 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:49.975947 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:49.975948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:49.975948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:49.976149 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:49.976234 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:49.976332 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 23:08:49.976425 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for oryx/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab.
Entry for principal oryx/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab.
I20260620 23:08:49.995144 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39667
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab
--principal=oryx/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:50.075768 9000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:50.075994 9000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:50.076036 9000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:50.077149 9000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:50.077214 9000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:50.077247 9000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 23:08:50.077277 9000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:50.078756 9000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab
--principal=oryx/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.32.126:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:50.079051 9000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:50.079283 9000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:50.084343 9008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:50.084352 9006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:50.084658 9000 server_base.cc:1034] running on GCE node
W20260620 23:08:50.084398 9005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:50.084945 9000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:50.085220 9000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:50.086405 9000 hybrid_clock.cc:648] HybridClock initialized: now 1781996930086360 us; error 57 us; skew 500 ppm
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996930, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:50.089126 9000 init.cc:377] Logged in from keytab as oryx/127.1.32.67@KRBTEST.COM (short username oryx)
I20260620 23:08:50.089520 9000 webserver.cc:460] Webserver started at http://127.1.32.67:40279/ using document root <none> and password file <none>
I20260620 23:08:50.089715 9000 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:50.089757 9000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:50.089849 9000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:50.090849 9000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "e07000a737d940558ed7ef7b25f08da4"
format_stamp: "Formatted at 2026-06-20 23:08:50 on dist-test-slave-fzjm"
server_key: "1f1e1cc576763693c839123ec673ba94"
server_key_iv: "efb8e022de2a94366c1a3c34b9ef33f2"
server_key_version: "encryptionkey@0"
I20260620 23:08:50.091239 9000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "e07000a737d940558ed7ef7b25f08da4"
format_stamp: "Formatted at 2026-06-20 23:08:50 on dist-test-slave-fzjm"
server_key: "1f1e1cc576763693c839123ec673ba94"
server_key_iv: "efb8e022de2a94366c1a3c34b9ef33f2"
server_key_version: "encryptionkey@0"
I20260620 23:08:50.092667 9000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.094017 9015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.094179 9000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.094297 9000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "e07000a737d940558ed7ef7b25f08da4"
format_stamp: "Formatted at 2026-06-20 23:08:50 on dist-test-slave-fzjm"
server_key: "1f1e1cc576763693c839123ec673ba94"
server_key_iv: "efb8e022de2a94366c1a3c34b9ef33f2"
server_key_version: "encryptionkey@0"
I20260620 23:08:50.094424 9000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:50.119791 9000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:50.122201 9000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:50.122362 9000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:50.122857 9000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:50.122936 9000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.122987 9000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:50.123018 9000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996930, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:50.130693 9000 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39173
I20260620 23:08:50.130784 9130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39173 every 8 connection(s)
I20260620 23:08:50.131139 9000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:08:50.132723 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.123384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:33465 (local address 127.1.32.126:39667)
0620 23:08:50.123441 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.123444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.123626 (+ 182us) server_negotiation.cc:407] Connection header received
0620 23:08:50.124115 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.124117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.124130 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.124218 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:50.124885 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.125678 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.127293 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.127544 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.127621 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.130104 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:50.130105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:50.130105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:50.130113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:50.131696 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.132047 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.132048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.132048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.132096 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.132264 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.132264 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.132264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.132355 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:50.132410 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.132605 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.132663 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:08:50.132941 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.123364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39667 (local address 127.1.32.67:33465)
0620 23:08:50.123493 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.123528 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.124018 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.124283 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.124286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.124533 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:50.124791 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.124796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.125801 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.125802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.127217 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.127220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.127643 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.127645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.127764 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.128242 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:50.128251 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:50.130017 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:50.131765 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:50.131768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:50.131770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:50.131982 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:50.132136 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:50.132137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:50.132137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:50.132218 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:50.132406 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:50.132408 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:50.132506 (+ 98us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.132590 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.132701 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260620 23:08:50.133340 9131 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39667
I20260620 23:08:50.133414 9131 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:50.133569 9131 heartbeater.cc:511] Master 127.1.32.126:39667 requested a full tablet report, sending...
I20260620 23:08:50.133929 8648 ts_manager.cc:194] Registered new tserver with Master: e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:50.134335 8648 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:33465
I20260620 23:08:50.138845 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.134346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:45801 (local address 127.1.32.126:39667)
0620 23:08:50.134389 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.134390 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.134482 (+ 92us) server_negotiation.cc:407] Connection header received
0620 23:08:50.134554 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.134555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.134565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.134624 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:50.134969 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.135440 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.136070 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.136235 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.136289 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.136984 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:50.136985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:50.136985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:50.136990 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:50.138073 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.138395 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.138396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.138396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.138439 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.138556 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.138556 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.138556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.138629 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:50.138669 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.138747 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.138796 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:08:50.138935 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.134273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39667 (local address 127.1.32.67:45801)
0620 23:08:50.134430 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.134441 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.134496 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.134628 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.134628 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.134775 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:50.134827 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.134828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.135519 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.135519 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.136002 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.136003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.136297 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.136298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.136350 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.136656 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:50.136659 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:50.136918 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:50.138151 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:50.138151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:50.138152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:50.138310 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:50.138479 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:50.138479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:50.138479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:50.138523 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:50.138661 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:50.138661 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:50.138691 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.138735 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.138804 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260620 23:08:50.139320 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 9000
I20260620 23:08:50.139443 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:50.139643 1153 external_mini_cluster.cc:1428] Setting key 353436ef5c5c1cb9e2133814ec5990be
I20260620 23:08:50.140422 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996930, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:08:50.159922 8700 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:50.154984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.1.32.126:39667)
0620 23:08:50.155036 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.155038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.155047 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:50.155073 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.155073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.155081 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.155131 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:50.155647 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.156101 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.156825 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.157068 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.157123 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.159833 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53330: BlockingRecv error: recv got EOF from 127.0.0.1:53330 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996930, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:50.166720 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.160565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.1.32.126:39667)
0620 23:08:50.160610 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.160612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.160704 (+ 92us) server_negotiation.cc:407] Connection header received
0620 23:08:50.160771 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.160772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.160780 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.160825 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:50.161350 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.161811 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.162441 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.162623 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.162689 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.164656 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:50.164657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:50.164672 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:50.164679 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:50.165736 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.166049 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.166050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.166051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.166102 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.166281 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.166282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.166282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.166379 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:50.166473 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.166562 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.166630 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:08:50.168186 8648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:50.170159 8648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:50.176327 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.172447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39173 (local address 127.0.0.1:46998)
0620 23:08:50.173036 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.173066 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.173120 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.173408 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.173409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.173411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:50.173460 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.173461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.174303 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.174304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.175287 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.175289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.176031 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.176032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.176106 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.176121 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.176203 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.176252 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":34,"threads_started":1}
I20260620 23:08:50.176342 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.172567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.1.32.67:39173)
0620 23:08:50.172699 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.172703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.173191 (+ 488us) server_negotiation.cc:407] Connection header received
0620 23:08:50.173240 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.173243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.173260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.173338 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:50.173545 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.174196 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.175434 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.175950 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.176021 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.176098 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.176213 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.176252 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260620 23:08:50.177351 9061 tablet_service.cc:1467] Processing CreateTablet for tablet 809536dcf16f435a97e9c106ee4cbf29 (DEFAULT_TABLE table=test-table [id=f1ce5608ac7643e189242f6c165e92b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.177747 9061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809536dcf16f435a97e9c106ee4cbf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.178076 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.172241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:36713 (local address 127.0.0.1:39726)
0620 23:08:50.172586 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.172629 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.172708 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.172971 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.172974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.172979 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:50.173055 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.173057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.174990 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.174991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.175793 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.175795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.177834 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.177834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.177905 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.177909 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.177967 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.178005 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":12,"thread_start_us":51,"threads_started":1}
I20260620 23:08:50.178102 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.172372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.1.32.66:36713)
0620 23:08:50.172569 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.172574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.172796 (+ 222us) server_negotiation.cc:407] Connection header received
0620 23:08:50.172846 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.172849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.172867 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.172957 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:50.173147 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.174252 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.175865 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.176572 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.177831 (+ 1259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.177909 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.177980 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.178012 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260620 23:08:50.178661 8925 tablet_service.cc:1467] Processing CreateTablet for tablet 809536dcf16f435a97e9c106ee4cbf29 (DEFAULT_TABLE table=test-table [id=f1ce5608ac7643e189242f6c165e92b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.178964 8925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809536dcf16f435a97e9c106ee4cbf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.181056 9156 tablet_bootstrap.cc:492] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Bootstrap starting.
I20260620 23:08:50.181679 9156 tablet_bootstrap.cc:654] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.182004 9156 log.cc:824] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:50.182155 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.172247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:38029 (local address 127.0.0.1:52172)
0620 23:08:50.172720 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.172729 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.172784 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.172971 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.172974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.172976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:50.173055 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.173058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.175874 (+ 2816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.175875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.176844 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.176848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.181846 (+ 4998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.181847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.181927 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.181932 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.182018 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.182053 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":54,"threads_started":1}
I20260620 23:08:50.182200 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.172588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52172 (local address 127.1.32.65:38029)
0620 23:08:50.172792 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.172796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.172807 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:50.172858 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.172860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.172875 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.172958 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:50.173147 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.174248 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.181112 (+ 6864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.181658 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.181699 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.181772 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.182040 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.182089 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":95,"threads_started":1}
I20260620 23:08:50.182410 9157 tablet_bootstrap.cc:492] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Bootstrap starting.
I20260620 23:08:50.182796 8785 tablet_service.cc:1467] Processing CreateTablet for tablet 809536dcf16f435a97e9c106ee4cbf29 (DEFAULT_TABLE table=test-table [id=f1ce5608ac7643e189242f6c165e92b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.183095 9156 tablet_bootstrap.cc:492] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: No bootstrap required, opened a new log
I20260620 23:08:50.183097 9157 tablet_bootstrap.cc:654] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.183166 8785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809536dcf16f435a97e9c106ee4cbf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.183166 9156 ts_tablet_manager.cc:1397] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:50.183409 9157 log.cc:824] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:50.184158 9157 tablet_bootstrap.cc:492] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: No bootstrap required, opened a new log
I20260620 23:08:50.184240 9157 ts_tablet_manager.cc:1397] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:50.185092 9156 raft_consensus.cc:348] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.185250 9156 raft_consensus.cc:374] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.185292 9156 raft_consensus.cc:729] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07000a737d940558ed7ef7b25f08da4, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.185433 9156 consensus_queue.cc:260] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.185753 9156 ts_tablet_manager.cc:1428] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:08:50.185781 9131 heartbeater.cc:503] Master 127.1.32.126:39667 was elected leader, sending a full tablet report...
I20260620 23:08:50.185824 9157 raft_consensus.cc:348] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.185974 9157 raft_consensus.cc:374] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.186013 9157 raft_consensus.cc:729] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afb552a650f457c99fac04f93ae69b0, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.186143 9157 consensus_queue.cc:260] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.186422 9157 ts_tablet_manager.cc:1428] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:08:50.186452 8991 heartbeater.cc:503] Master 127.1.32.126:39667 was elected leader, sending a full tablet report...
I20260620 23:08:50.186815 9162 tablet_bootstrap.cc:492] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Bootstrap starting.
I20260620 23:08:50.187454 9162 tablet_bootstrap.cc:654] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.187736 9162 log.cc:824] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:50.188463 9162 tablet_bootstrap.cc:492] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: No bootstrap required, opened a new log
I20260620 23:08:50.188517 9162 ts_tablet_manager.cc:1397] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:08:50.189934 9162 raft_consensus.cc:348] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.190040 9162 raft_consensus.cc:374] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.190091 9162 raft_consensus.cc:729] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.190198 9162 consensus_queue.cc:260] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.190424 9162 ts_tablet_manager.cc:1428] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 23:08:50.190478 8851 heartbeater.cc:503] Master 127.1.32.126:39667 was elected leader, sending a full tablet report...
W20260620 23:08:50.216331 8992 tablet.cc:2369] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:50.298583 9160 raft_consensus.cc:482] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:50.298715 9160 raft_consensus.cc:504] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.299078 9160 leader_election.cc:288] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029)
I20260620 23:08:50.301844 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.299413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:60589 (local address 127.1.32.65:38029)
0620 23:08:50.299475 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.299477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.299483 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:08:50.299509 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.299510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.299519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.299598 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:50.299755 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.300224 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.301140 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.301512 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.301584 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.301594 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.301741 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.301776 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 23:08:50.301844 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.299286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:38029 (local address 127.1.32.67:60589)
0620 23:08:50.299327 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.299339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.299383 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.299604 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.299605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.299607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:50.299677 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.299679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.300301 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.300302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.301052 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.301054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.301594 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.301595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.301675 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.301678 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.301736 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.301774 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 23:08:50.302096 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.299286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:36713 (local address 127.1.32.67:52967)
0620 23:08:50.299604 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.299613 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.299689 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.299833 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.299833 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.299835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:50.299906 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.299912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.300515 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.300516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.301236 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.301237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.301756 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.301757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.301865 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.301868 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.301932 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.301976 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":146,"thread_start_us":122,"threads_started":1}
I20260620 23:08:50.302179 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.299350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:52967 (local address 127.1.32.66:36713)
0620 23:08:50.299398 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.299400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.299679 (+ 279us) server_negotiation.cc:407] Connection header received
0620 23:08:50.299743 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.299744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.299757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.299818 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:50.299973 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.300433 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.301327 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.301682 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.301755 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.301764 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.302063 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.302106 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:08:50.302189 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809536dcf16f435a97e9c106ee4cbf29" candidate_uuid: "e07000a737d940558ed7ef7b25f08da4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0a5708a012449ab80fa30286c7f82e" is_pre_election: true
I20260620 23:08:50.302346 8805 raft_consensus.cc:2454] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e07000a737d940558ed7ef7b25f08da4 in term 0.
I20260620 23:08:50.302480 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809536dcf16f435a97e9c106ee4cbf29" candidate_uuid: "e07000a737d940558ed7ef7b25f08da4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0" is_pre_election: true
I20260620 23:08:50.302587 9016 leader_election.cc:302] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0a5708a012449ab80fa30286c7f82e, e07000a737d940558ed7ef7b25f08da4; no voters:
I20260620 23:08:50.302644 8945 raft_consensus.cc:2454] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e07000a737d940558ed7ef7b25f08da4 in term 0.
I20260620 23:08:50.302716 9160 raft_consensus.cc:2781] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:50.302762 9160 raft_consensus.cc:482] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:50.302778 9160 raft_consensus.cc:3037] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.303714 9160 raft_consensus.cc:504] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.303855 9160 leader_election.cc:288] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [CANDIDATE]: Term 1 election: Requested vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029)
I20260620 23:08:50.304018 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809536dcf16f435a97e9c106ee4cbf29" candidate_uuid: "e07000a737d940558ed7ef7b25f08da4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0"
I20260620 23:08:50.304018 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809536dcf16f435a97e9c106ee4cbf29" candidate_uuid: "e07000a737d940558ed7ef7b25f08da4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0a5708a012449ab80fa30286c7f82e"
I20260620 23:08:50.304107 8805 raft_consensus.cc:3037] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.304107 8945 raft_consensus.cc:3037] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.304948 8805 raft_consensus.cc:2454] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e07000a737d940558ed7ef7b25f08da4 in term 1.
I20260620 23:08:50.304949 8945 raft_consensus.cc:2454] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e07000a737d940558ed7ef7b25f08da4 in term 1.
I20260620 23:08:50.305101 9016 leader_election.cc:302] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0a5708a012449ab80fa30286c7f82e, e07000a737d940558ed7ef7b25f08da4; no voters:
I20260620 23:08:50.305224 9160 raft_consensus.cc:2781] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:50.305384 9160 raft_consensus.cc:686] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 1 LEADER]: Becoming Leader. State: Replica: e07000a737d940558ed7ef7b25f08da4, State: Running, Role: LEADER
I20260620 23:08:50.305472 9160 consensus_queue.cc:237] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.306284 8647 catalog_manager.cc:5495] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e07000a737d940558ed7ef7b25f08da4 (127.1.32.67). New cstate: current_term: 1 leader_uuid: "e07000a737d940558ed7ef7b25f08da4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } health_report { overall_health: HEALTHY } } }
I20260620 23:08:50.319119 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.317167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.1.32.67:39173)
0620 23:08:50.317227 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.317228 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.317235 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:50.317297 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.317298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.317311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.317382 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:50.317471 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.317889 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.318490 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.318646 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.318691 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.318798 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:50.318877 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:50.318940 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.319013 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.319027 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 23:08:50.320660 8945 raft_consensus.cc:1261] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Refusing update from remote peer e07000a737d940558ed7ef7b25f08da4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.320673 8805 raft_consensus.cc:1261] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Refusing update from remote peer e07000a737d940558ed7ef7b25f08da4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.320966 9166 consensus_queue.cc:1035] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.321094 9160 consensus_queue.cc:1035] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.325598 8647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:50.325709 8647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:50.326380 8647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Sending DeleteTablet for 3 replicas of tablet 809536dcf16f435a97e9c106ee4cbf29
I20260620 23:08:50.326689 8925 tablet_service.cc:1514] Processing DeleteTablet for tablet 809536dcf16f435a97e9c106ee4cbf29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39726
I20260620 23:08:50.326763 8785 tablet_service.cc:1514] Processing DeleteTablet for tablet 809536dcf16f435a97e9c106ee4cbf29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:52172
I20260620 23:08:50.326968 9061 tablet_service.cc:1514] Processing DeleteTablet for tablet 809536dcf16f435a97e9c106ee4cbf29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:46998
I20260620 23:08:50.327020 9180 tablet_replica.cc:330] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: stopping tablet replica
I20260620 23:08:50.327106 9180 raft_consensus.cc:2229] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:50.327219 9180 raft_consensus.cc:2258] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.327162 8647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:50.327322 8647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:50.327232 9181 tablet_replica.cc:330] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: stopping tablet replica
I20260620 23:08:50.327656 9181 raft_consensus.cc:2229] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:50.327750 9180 ts_tablet_manager.cc:1905] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.327853 9181 raft_consensus.cc:2258] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.328346 9181 ts_tablet_manager.cc:1905] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.328668 8925 tablet_service.cc:1467] Processing CreateTablet for tablet 3b643de872a14073827b2d9ca1b44612 (DEFAULT_TABLE table=test-table [id=94567cfe90fc4a01b763e1ff9b931910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.328783 8925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b643de872a14073827b2d9ca1b44612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.329284 9180 ts_tablet_manager.cc:1918] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:50.329367 9180 log.cc:1196] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/809536dcf16f435a97e9c106ee4cbf29
I20260620 23:08:50.329623 9180 ts_tablet_manager.cc:1939] T 809536dcf16f435a97e9c106ee4cbf29 P 3f0a5708a012449ab80fa30286c7f82e: Deleting consensus metadata
I20260620 23:08:50.330284 9181 ts_tablet_manager.cc:1918] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:50.330384 9181 log.cc:1196] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/809536dcf16f435a97e9c106ee4cbf29
I20260620 23:08:50.330431 9157 tablet_bootstrap.cc:492] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: Bootstrap starting.
I20260620 23:08:50.330627 9181 ts_tablet_manager.cc:1939] T 809536dcf16f435a97e9c106ee4cbf29 P e07000a737d940558ed7ef7b25f08da4: Deleting consensus metadata
I20260620 23:08:50.330847 9157 tablet_bootstrap.cc:654] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.331135 8632 catalog_manager.cc:4841] TS 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029): tablet 809536dcf16f435a97e9c106ee4cbf29 (table test-table [id=f1ce5608ac7643e189242f6c165e92b7]) successfully deleted
I20260620 23:08:50.331219 8632 catalog_manager.cc:4841] TS e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173): tablet 809536dcf16f435a97e9c106ee4cbf29 (table test-table [id=f1ce5608ac7643e189242f6c165e92b7]) successfully deleted
I20260620 23:08:50.331364 8785 tablet_service.cc:1467] Processing CreateTablet for tablet 3b643de872a14073827b2d9ca1b44612 (DEFAULT_TABLE table=test-table [id=94567cfe90fc4a01b763e1ff9b931910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.331480 8785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b643de872a14073827b2d9ca1b44612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.331487 9061 tablet_service.cc:1467] Processing CreateTablet for tablet 3b643de872a14073827b2d9ca1b44612 (DEFAULT_TABLE table=test-table [id=94567cfe90fc4a01b763e1ff9b931910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.331565 9061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b643de872a14073827b2d9ca1b44612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.331727 9179 tablet_replica.cc:330] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: stopping tablet replica
I20260620 23:08:50.331851 9179 raft_consensus.cc:2229] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:50.331995 9179 raft_consensus.cc:2258] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.332463 9179 ts_tablet_manager.cc:1905] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.332649 9162 tablet_bootstrap.cc:492] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Bootstrap starting.
I20260620 23:08:50.333233 9162 tablet_bootstrap.cc:654] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.333554 9156 tablet_bootstrap.cc:492] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Bootstrap starting.
I20260620 23:08:50.333931 9156 tablet_bootstrap.cc:654] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.334370 9179 ts_tablet_manager.cc:1918] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:50.334453 9162 tablet_bootstrap.cc:492] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: No bootstrap required, opened a new log
I20260620 23:08:50.334465 9179 log.cc:1196] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/809536dcf16f435a97e9c106ee4cbf29
I20260620 23:08:50.334501 9162 ts_tablet_manager.cc:1397] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:08:50.334640 9162 raft_consensus.cc:348] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.334710 9162 raft_consensus.cc:374] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.334754 9162 raft_consensus.cc:729] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.334803 9162 consensus_queue.cc:260] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.334919 9162 ts_tablet_manager.cc:1428] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:50.335165 9156 tablet_bootstrap.cc:492] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: No bootstrap required, opened a new log
I20260620 23:08:50.335223 9156 ts_tablet_manager.cc:1397] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:50.335378 9179 ts_tablet_manager.cc:1939] T 809536dcf16f435a97e9c106ee4cbf29 P 1afb552a650f457c99fac04f93ae69b0: Deleting consensus metadata
I20260620 23:08:50.335369 9156 raft_consensus.cc:348] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.335433 9156 raft_consensus.cc:374] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.335464 9156 raft_consensus.cc:729] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07000a737d940558ed7ef7b25f08da4, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.335508 9156 consensus_queue.cc:260] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.335604 9156 ts_tablet_manager.cc:1428] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:50.336027 9157 tablet_bootstrap.cc:492] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: No bootstrap required, opened a new log
I20260620 23:08:50.336050 8633 catalog_manager.cc:4841] TS 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713): tablet 809536dcf16f435a97e9c106ee4cbf29 (table test-table [id=f1ce5608ac7643e189242f6c165e92b7]) successfully deleted
I20260620 23:08:50.336079 9157 ts_tablet_manager.cc:1397] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260620 23:08:50.336208 9157 raft_consensus.cc:348] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.336293 9157 raft_consensus.cc:374] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.336318 9157 raft_consensus.cc:729] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afb552a650f457c99fac04f93ae69b0, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.336357 9157 consensus_queue.cc:260] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.336452 9157 ts_tablet_manager.cc:1428] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260620 23:08:50.381662 9132 tablet.cc:2369] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:08:50.541518 9164 raft_consensus.cc:482] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:50.541630 9164 raft_consensus.cc:504] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.542006 9164 leader_election.cc:288] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:50.545254 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.542209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:36049 (local address 127.1.32.66:36713)
0620 23:08:50.542275 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.542277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.542380 (+ 103us) server_negotiation.cc:407] Connection header received
0620 23:08:50.542473 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.542474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.542487 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.542554 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:50.542746 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.543510 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.544544 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.544895 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.544965 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.544976 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.545128 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.545179 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:08:50.545276 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.542244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:38117 (local address 127.1.32.67:39173)
0620 23:08:50.542292 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.542294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.542433 (+ 139us) server_negotiation.cc:407] Connection header received
0620 23:08:50.542545 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.542546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.542556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.542610 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:50.542785 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.543520 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.544517 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.544897 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.544980 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.544990 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.545171 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.545208 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:08:50.545290 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.542145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:36713 (local address 127.1.32.65:36049)
0620 23:08:50.542292 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.542304 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.542390 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.542566 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.542567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.542572 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:50.542652 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.542655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.543584 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.543585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.544437 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.544439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.544979 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.544980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.545060 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.545063 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.545111 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.545182 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260620 23:08:50.545303 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.542190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39173 (local address 127.1.32.65:38117)
0620 23:08:50.542328 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.542344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.542449 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.542627 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.542628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.542630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:50.542722 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.542724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.543620 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.543621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.544429 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.544431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.545029 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.545030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.545075 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.545078 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.545145 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.545192 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260620 23:08:50.545560 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b643de872a14073827b2d9ca1b44612" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0" is_pre_election: true
I20260620 23:08:50.545660 8945 raft_consensus.cc:2454] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 0.
I20260620 23:08:50.545701 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b643de872a14073827b2d9ca1b44612" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4" is_pre_election: true
I20260620 23:08:50.545842 9085 raft_consensus.cc:2454] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 0.
I20260620 23:08:50.545835 8737 leader_election.cc:302] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1afb552a650f457c99fac04f93ae69b0, 3f0a5708a012449ab80fa30286c7f82e; no voters:
I20260620 23:08:50.545969 9164 raft_consensus.cc:2781] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:50.546013 9164 raft_consensus.cc:482] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:50.546041 9164 raft_consensus.cc:3037] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.546687 9164 raft_consensus.cc:504] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.546818 9164 leader_election.cc:288] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 election: Requested vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:50.547000 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b643de872a14073827b2d9ca1b44612" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0"
I20260620 23:08:50.547019 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b643de872a14073827b2d9ca1b44612" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4"
I20260620 23:08:50.547078 8945 raft_consensus.cc:3037] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.547098 9085 raft_consensus.cc:3037] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.547713 9085 raft_consensus.cc:2454] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 1.
I20260620 23:08:50.547870 8945 raft_consensus.cc:2454] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 1.
I20260620 23:08:50.547892 8736 leader_election.cc:302] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0a5708a012449ab80fa30286c7f82e, e07000a737d940558ed7ef7b25f08da4; no voters:
I20260620 23:08:50.547964 9164 raft_consensus.cc:2781] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:50.548070 9164 raft_consensus.cc:686] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 1 LEADER]: Becoming Leader. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Running, Role: LEADER
I20260620 23:08:50.548141 9164 consensus_queue.cc:237] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.548821 8647 catalog_manager.cc:5495] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65). New cstate: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } health_report { overall_health: UNKNOWN } } }
W20260620 23:08:50.575052 8852 tablet.cc:2369] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996929, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.126@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:50.602092 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.595524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39667 (local address 127.0.0.1:53348)
0620 23:08:50.595676 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.595685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.595764 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.595919 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.595920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.596217 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:50.596258 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.596260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.596965 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.596966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.597470 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.597471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.597845 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.597846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.597896 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.598409 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:50.598414 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:50.599842 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:50.601088 (+ 1246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:50.601094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:50.601097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:50.601283 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:50.601469 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:50.601470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:50.601470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:50.601524 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:50.601733 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:50.601739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:50.601781 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.601853 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.601962 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260620 23:08:50.602092 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.595562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.1.32.126:39667)
0620 23:08:50.595624 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.595626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.595745 (+ 119us) server_negotiation.cc:407] Connection header received
0620 23:08:50.595830 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.595831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.595841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.595915 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:50.596343 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.596868 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.597598 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.597773 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.597842 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.599922 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:50.599923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:50.599923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:50.599929 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:50.601006 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.601360 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.601361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.601362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.601411 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.601580 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.601580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.601581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.601676 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:50.601742 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.601893 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.602007 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":35}
I20260620 23:08:50.603950 8647 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:53348:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 23:08:50.604177 8647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:50.605521 9061 tablet_service.cc:1467] Processing CreateTablet for tablet 6956fe6e27214a0eb32737532b2cb4bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afd25b4f3309469cbd36ecd373427338]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:50.605521 8785 tablet_service.cc:1467] Processing CreateTablet for tablet 6956fe6e27214a0eb32737532b2cb4bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afd25b4f3309469cbd36ecd373427338]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:50.605688 9061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6956fe6e27214a0eb32737532b2cb4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.605688 8785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6956fe6e27214a0eb32737532b2cb4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.607043 8925 tablet_service.cc:1467] Processing CreateTablet for tablet 6956fe6e27214a0eb32737532b2cb4bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afd25b4f3309469cbd36ecd373427338]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 23:08:50.607167 9156 tablet_bootstrap.cc:492] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: Bootstrap starting.
I20260620 23:08:50.607188 8925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6956fe6e27214a0eb32737532b2cb4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.607506 9156 tablet_bootstrap.cc:654] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.607756 9162 tablet_bootstrap.cc:492] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: Bootstrap starting.
I20260620 23:08:50.608017 9156 tablet_bootstrap.cc:492] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: No bootstrap required, opened a new log
I20260620 23:08:50.608049 9156 ts_tablet_manager.cc:1397] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.608141 9162 tablet_bootstrap.cc:654] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.608134 9156 raft_consensus.cc:348] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.608175 9156 raft_consensus.cc:374] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.608189 9156 raft_consensus.cc:729] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07000a737d940558ed7ef7b25f08da4, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.608242 9156 consensus_queue.cc:260] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.608340 9156 ts_tablet_manager.cc:1428] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.608353 9176 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.608428 9176 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:50.608776 9157 tablet_bootstrap.cc:492] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: Bootstrap starting.
I20260620 23:08:50.609165 9157 tablet_bootstrap.cc:654] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.609258 9162 tablet_bootstrap.cc:492] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: No bootstrap required, opened a new log
I20260620 23:08:50.609308 9162 ts_tablet_manager.cc:1397] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:08:50.609447 9162 raft_consensus.cc:348] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.609524 9162 raft_consensus.cc:374] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.609548 9162 raft_consensus.cc:729] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.609587 9162 consensus_queue.cc:260] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.609689 9162 ts_tablet_manager.cc:1428] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:50.609786 9157 tablet_bootstrap.cc:492] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: No bootstrap required, opened a new log
I20260620 23:08:50.609851 9157 ts_tablet_manager.cc:1397] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.609825 9164 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.609889 9164 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:50.609975 9157 raft_consensus.cc:348] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.610042 9157 raft_consensus.cc:374] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.610062 9157 raft_consensus.cc:729] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afb552a650f457c99fac04f93ae69b0, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.610103 9157 consensus_queue.cc:260] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.610208 9157 ts_tablet_manager.cc:1428] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.610230 9161 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.610299 9161 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:50.622988 9164 raft_consensus.cc:482] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:50.623040 9164 raft_consensus.cc:504] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.623181 9164 leader_election.cc:288] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:50.623318 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6956fe6e27214a0eb32737532b2cb4bf" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0" is_pre_election: true
I20260620 23:08:50.623334 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6956fe6e27214a0eb32737532b2cb4bf" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4" is_pre_election: true
I20260620 23:08:50.623394 8945 raft_consensus.cc:2454] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 0.
I20260620 23:08:50.623396 9085 raft_consensus.cc:2454] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 0.
I20260620 23:08:50.623522 8737 leader_election.cc:302] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1afb552a650f457c99fac04f93ae69b0, 3f0a5708a012449ab80fa30286c7f82e; no voters:
I20260620 23:08:50.623651 9164 raft_consensus.cc:2781] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:50.623697 9164 raft_consensus.cc:482] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:50.623711 9164 raft_consensus.cc:3037] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.624188 9164 raft_consensus.cc:504] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.624301 9164 leader_election.cc:288] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 election: Requested vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:50.624430 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6956fe6e27214a0eb32737532b2cb4bf" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0"
I20260620 23:08:50.624454 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6956fe6e27214a0eb32737532b2cb4bf" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4"
I20260620 23:08:50.624497 8945 raft_consensus.cc:3037] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.624516 9085 raft_consensus.cc:3037] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.624985 8945 raft_consensus.cc:2454] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 1.
I20260620 23:08:50.625034 9085 raft_consensus.cc:2454] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 1.
I20260620 23:08:50.625177 8737 leader_election.cc:302] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1afb552a650f457c99fac04f93ae69b0, 3f0a5708a012449ab80fa30286c7f82e; no voters:
I20260620 23:08:50.625265 9164 raft_consensus.cc:2781] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:50.625330 9164 raft_consensus.cc:686] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [term 1 LEADER]: Becoming Leader. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Running, Role: LEADER
I20260620 23:08:50.625385 9164 consensus_queue.cc:237] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.625633 9187 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: TxnStatusTablet state changed. Reason: New leader 3f0a5708a012449ab80fa30286c7f82e. Latest consensus state: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.625734 9187 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:50.625833 8647 catalog_manager.cc:5495] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65). New cstate: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:50.625898 9199 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 23:08:50.625979 9199 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 23:08:50.632550 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.630240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.1.32.65:38029)
0620 23:08:50.630274 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.630276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.630332 (+ 56us) server_negotiation.cc:407] Connection header received
0620 23:08:50.630409 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.630410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.630426 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.630482 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:50.630567 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.631103 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.631775 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.631982 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.632053 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.632215 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:50.632307 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:50.632418 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.632471 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.632489 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 23:08:50.632575 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.630193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:38029 (local address 127.0.0.1:52186)
0620 23:08:50.630292 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.630302 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.630350 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.630485 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.630486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.630487 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:50.630523 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.630525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.631180 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.631181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.631689 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.631691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.632055 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.632056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.632112 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.632124 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:50.632407 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:50.632414 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.632467 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.632506 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260620 23:08:50.633742 8945 raft_consensus.cc:1261] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Refusing update from remote peer 3f0a5708a012449ab80fa30286c7f82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.633826 9085 raft_consensus.cc:1261] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Refusing update from remote peer 3f0a5708a012449ab80fa30286c7f82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.633963 9187 consensus_queue.cc:1035] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.634102 9164 consensus_queue.cc:1035] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.634456 9161 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: TxnStatusTablet state changed. Reason: New leader 3f0a5708a012449ab80fa30286c7f82e. Latest consensus state: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.634543 9161 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:50.634557 9176 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: TxnStatusTablet state changed. Reason: New leader 3f0a5708a012449ab80fa30286c7f82e. Latest consensus state: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.634625 9176 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:50.635202 9164 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.635258 9164 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:50.635270 9170 mvcc.cc:205] Tried to move back new op lower bound from 7299059427874078720 to 7299059427841880064. Current Snapshot: MvccSnapshot[applied={T|T < 7299059427874078720 or (T in {7299059427874078720})}]
I20260620 23:08:50.635181 9187 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.635421 9187 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P 3f0a5708a012449ab80fa30286c7f82e: This TxnStatusTablet replica's current role is: LEADER
I20260620 23:08:50.636077 9171 mvcc.cc:205] Tried to move back new op lower bound from 7299059427874078720 to 7299059427841880064. Current Snapshot: MvccSnapshot[applied={T|T < 7299059427874078720}]
I20260620 23:08:50.636158 9169 mvcc.cc:205] Tried to move back new op lower bound from 7299059427874078720 to 7299059427841880064. Current Snapshot: MvccSnapshot[applied={T|T < 7299059427874078720}]
I20260620 23:08:50.636173 9161 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.636231 9161 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P 1afb552a650f457c99fac04f93ae69b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:50.636307 9176 tablet_replica.cc:439] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } } }
I20260620 23:08:50.636369 9176 tablet_replica.cc:442] T 6956fe6e27214a0eb32737532b2cb4bf P e07000a737d940558ed7ef7b25f08da4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 23:08:50.638866 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.636796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52192 (local address 127.1.32.65:38029)
0620 23:08:50.636829 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.636831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.636837 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:08:50.636862 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.636863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.636874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.636916 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:50.637055 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.637491 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.638192 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.638388 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.638464 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.638627 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:50.638656 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:50.638709 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.638768 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.638784 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:08:50.643517 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.640986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:42563 (local address 127.1.32.65:38029)
0620 23:08:50.641020 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.641022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.641162 (+ 140us) server_negotiation.cc:407] Connection header received
0620 23:08:50.641187 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.641187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.641195 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.641255 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:50.641329 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.641951 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.642729 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.643065 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.643125 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.643289 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:50.643317 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:50.643383 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.643431 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.643446 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:08:50.643600 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.640945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:38029 (local address 127.1.32.65:42563)
0620 23:08:50.641103 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:50.641112 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:50.641157 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:50.641256 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:50.641256 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:50.641257 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:50.641298 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.641298 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.642005 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.642006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.642668 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:50.642669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.643137 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.643138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:50.643211 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.643222 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:50.643383 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:50.643384 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:08:50.643413 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 23:08:50.643469 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260620 23:08:50.645226 8945 raft_consensus.cc:1261] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Refusing update from remote peer 3f0a5708a012449ab80fa30286c7f82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.645366 9085 raft_consensus.cc:1261] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Refusing update from remote peer 3f0a5708a012449ab80fa30286c7f82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.645500 9187 consensus_queue.cc:1035] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.645655 9187 consensus_queue.cc:1035] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.658577 8648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:50.658655 8648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:50.659200 8648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Sending DeleteTablet for 3 replicas of tablet 3b643de872a14073827b2d9ca1b44612
I20260620 23:08:50.659432 8925 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b643de872a14073827b2d9ca1b44612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39726
I20260620 23:08:50.659444 8784 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b643de872a14073827b2d9ca1b44612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:52172
I20260620 23:08:50.659437 9061 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b643de872a14073827b2d9ca1b44612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:46998
I20260620 23:08:50.659523 9181 tablet_replica.cc:330] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: stopping tablet replica
I20260620 23:08:50.659523 9179 tablet_replica.cc:330] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: stopping tablet replica
I20260620 23:08:50.659577 9179 raft_consensus.cc:2229] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:50.659583 9181 raft_consensus.cc:2229] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:50.659605 9179 raft_consensus.cc:2258] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.659613 9181 raft_consensus.cc:2258] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.659678 9180 tablet_replica.cc:330] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: stopping tablet replica
I20260620 23:08:50.659723 9180 raft_consensus.cc:2229] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:50.659799 9180 raft_consensus.cc:2258] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.659929 9181 ts_tablet_manager.cc:1905] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.659929 9179 ts_tablet_manager.cc:1905] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.661593 9180 ts_tablet_manager.cc:1905] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.663093 9180 ts_tablet_manager.cc:1918] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:50.663139 9180 log.cc:1196] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/3b643de872a14073827b2d9ca1b44612
I20260620 23:08:50.663345 9180 ts_tablet_manager.cc:1939] T 3b643de872a14073827b2d9ca1b44612 P 3f0a5708a012449ab80fa30286c7f82e: Deleting consensus metadata
I20260620 23:08:50.663872 8632 catalog_manager.cc:4841] TS 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029): tablet 3b643de872a14073827b2d9ca1b44612 (table test-table [id=94567cfe90fc4a01b763e1ff9b931910]) successfully deleted
I20260620 23:08:50.663923 9179 ts_tablet_manager.cc:1918] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:50.663973 9179 log.cc:1196] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/3b643de872a14073827b2d9ca1b44612
I20260620 23:08:50.664048 9181 ts_tablet_manager.cc:1918] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:50.664115 9181 log.cc:1196] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/3b643de872a14073827b2d9ca1b44612
I20260620 23:08:50.664202 9179 ts_tablet_manager.cc:1939] T 3b643de872a14073827b2d9ca1b44612 P 1afb552a650f457c99fac04f93ae69b0: Deleting consensus metadata
I20260620 23:08:50.664372 9181 ts_tablet_manager.cc:1939] T 3b643de872a14073827b2d9ca1b44612 P e07000a737d940558ed7ef7b25f08da4: Deleting consensus metadata
I20260620 23:08:50.664491 8633 catalog_manager.cc:4841] TS 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713): tablet 3b643de872a14073827b2d9ca1b44612 (table test-table [id=94567cfe90fc4a01b763e1ff9b931910]) successfully deleted
I20260620 23:08:50.664650 8632 catalog_manager.cc:4841] TS e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173): tablet 3b643de872a14073827b2d9ca1b44612 (table test-table [id=94567cfe90fc4a01b763e1ff9b931910]) successfully deleted
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:08:50.682727 8700 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:50.676901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.1.32.126:39667)
0620 23:08:50.676992 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.676994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.677073 (+ 79us) server_negotiation.cc:407] Connection header received
0620 23:08:50.677137 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.677138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.677148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.677201 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:50.677699 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.678501 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.679521 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.679681 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.679749 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.682666 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53350: BlockingRecv error: recv got EOF from 127.0.0.1:53350 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 23:08:50 dist-test-slave-fzjm krb5kdc[8603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM
I20260620 23:08:50.690344 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.683437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.1.32.126:39667)
0620 23:08:50.683473 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.683474 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.683574 (+ 100us) server_negotiation.cc:407] Connection header received
0620 23:08:50.683627 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.683627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.683634 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.683670 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:50.684150 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.684547 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.685496 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.685660 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.685712 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.687794 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:50.687795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:50.687796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:50.687802 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:50.689182 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.689586 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.689588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.689588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.689646 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:50.689842 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:50.689843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:50.689843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:50.689954 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:50.690051 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.690152 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.690237 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:08:50.691730 8648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:08:50.691905 8648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:50.693099 9061 tablet_service.cc:1467] Processing CreateTablet for tablet 1c205601b875483db94eef8ccf3f8ff2 (DEFAULT_TABLE table=test-table [id=648d9df330f8423dbc48fa754bfc8258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.693122 8784 tablet_service.cc:1467] Processing CreateTablet for tablet 1c205601b875483db94eef8ccf3f8ff2 (DEFAULT_TABLE table=test-table [id=648d9df330f8423dbc48fa754bfc8258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.693218 9061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c205601b875483db94eef8ccf3f8ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.693218 8784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c205601b875483db94eef8ccf3f8ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.693205 8925 tablet_service.cc:1467] Processing CreateTablet for tablet 1c205601b875483db94eef8ccf3f8ff2 (DEFAULT_TABLE table=test-table [id=648d9df330f8423dbc48fa754bfc8258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.693312 8925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c205601b875483db94eef8ccf3f8ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.694478 9156 tablet_bootstrap.cc:492] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: Bootstrap starting.
I20260620 23:08:50.694528 9162 tablet_bootstrap.cc:492] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: Bootstrap starting.
I20260620 23:08:50.694775 9156 tablet_bootstrap.cc:654] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.694877 9162 tablet_bootstrap.cc:654] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.695191 9157 tablet_bootstrap.cc:492] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: Bootstrap starting.
I20260620 23:08:50.695478 9162 tablet_bootstrap.cc:492] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: No bootstrap required, opened a new log
I20260620 23:08:50.695557 9162 ts_tablet_manager.cc:1397] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.695618 9157 tablet_bootstrap.cc:654] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.695710 9162 raft_consensus.cc:348] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.695791 9162 raft_consensus.cc:374] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.695819 9162 raft_consensus.cc:729] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.695858 9162 consensus_queue.cc:260] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.695961 9162 ts_tablet_manager.cc:1428] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:50.696441 9157 tablet_bootstrap.cc:492] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: No bootstrap required, opened a new log
I20260620 23:08:50.696492 9157 ts_tablet_manager.cc:1397] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.696581 9156 tablet_bootstrap.cc:492] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: No bootstrap required, opened a new log
I20260620 23:08:50.696630 9156 ts_tablet_manager.cc:1397] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:08:50.696645 9157 raft_consensus.cc:348] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.696734 9157 raft_consensus.cc:374] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.696759 9157 raft_consensus.cc:729] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afb552a650f457c99fac04f93ae69b0, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.696760 9156 raft_consensus.cc:348] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.696800 9157 consensus_queue.cc:260] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.696833 9156 raft_consensus.cc:374] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.696851 9156 raft_consensus.cc:729] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07000a737d940558ed7ef7b25f08da4, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.696887 9157 ts_tablet_manager.cc:1428] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.696903 9156 consensus_queue.cc:260] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.696986 9156 ts_tablet_manager.cc:1428] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.729182 9164 raft_consensus.cc:482] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:50.729269 9164 raft_consensus.cc:504] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.729425 9164 leader_election.cc:288] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:50.729575 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c205601b875483db94eef8ccf3f8ff2" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0" is_pre_election: true
I20260620 23:08:50.729697 8945 raft_consensus.cc:2454] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 0.
I20260620 23:08:50.729671 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c205601b875483db94eef8ccf3f8ff2" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4" is_pre_election: true
I20260620 23:08:50.729784 9085 raft_consensus.cc:2454] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 0.
I20260620 23:08:50.729952 8736 leader_election.cc:302] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0a5708a012449ab80fa30286c7f82e, e07000a737d940558ed7ef7b25f08da4; no voters:
I20260620 23:08:50.730044 9164 raft_consensus.cc:2781] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:50.730082 9164 raft_consensus.cc:482] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:50.730108 9164 raft_consensus.cc:3037] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.730690 9164 raft_consensus.cc:504] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.730811 9164 leader_election.cc:288] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 election: Requested vote from peers 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:50.730965 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c205601b875483db94eef8ccf3f8ff2" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4"
I20260620 23:08:50.730994 8945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c205601b875483db94eef8ccf3f8ff2" candidate_uuid: "3f0a5708a012449ab80fa30286c7f82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1afb552a650f457c99fac04f93ae69b0"
I20260620 23:08:50.731037 9085 raft_consensus.cc:3037] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.731048 8945 raft_consensus.cc:3037] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:50.731695 9085 raft_consensus.cc:2454] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 1.
I20260620 23:08:50.731695 8945 raft_consensus.cc:2454] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0a5708a012449ab80fa30286c7f82e in term 1.
I20260620 23:08:50.731887 8736 leader_election.cc:302] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0a5708a012449ab80fa30286c7f82e, e07000a737d940558ed7ef7b25f08da4; no voters:
I20260620 23:08:50.731982 9164 raft_consensus.cc:2781] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:50.732034 9164 raft_consensus.cc:686] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 1 LEADER]: Becoming Leader. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Running, Role: LEADER
I20260620 23:08:50.732082 9164 consensus_queue.cc:237] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.732602 8648 catalog_manager.cc:5495] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65). New cstate: current_term: 1 leader_uuid: "3f0a5708a012449ab80fa30286c7f82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:50.745224 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:50.743055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52208 (local address 127.1.32.65:38029)
0620 23:08:50.743119 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:50.743121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:50.743131 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:50.743187 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:50.743188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:50.743198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:50.743250 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:50.743377 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.743877 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.744623 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:50.744797 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:50.744862 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:50.745001 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:50.745033 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:50.745085 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:50.745137 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 23:08:50.745153 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:08:50.745972 8945 raft_consensus.cc:1261] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Refusing update from remote peer 3f0a5708a012449ab80fa30286c7f82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.745987 9085 raft_consensus.cc:1261] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Refusing update from remote peer 3f0a5708a012449ab80fa30286c7f82e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:50.746199 9204 consensus_queue.cc:1035] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.746272 9164 consensus_queue.cc:1035] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:50.749133 8648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:50.749207 8648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:50.749756 8648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Sending DeleteTablet for 3 replicas of tablet 1c205601b875483db94eef8ccf3f8ff2
I20260620 23:08:50.749979 8784 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c205601b875483db94eef8ccf3f8ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:52172
I20260620 23:08:50.750036 8925 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c205601b875483db94eef8ccf3f8ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39726
I20260620 23:08:50.750062 9061 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c205601b875483db94eef8ccf3f8ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:50 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:46998
I20260620 23:08:50.750103 9179 tablet_replica.cc:330] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: stopping tablet replica
I20260620 23:08:50.750133 9180 tablet_replica.cc:330] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: stopping tablet replica
I20260620 23:08:50.750147 9179 raft_consensus.cc:2229] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:50.750175 9180 raft_consensus.cc:2229] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:50.750186 9179 raft_consensus.cc:2258] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.750221 9180 raft_consensus.cc:2258] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.750280 9181 tablet_replica.cc:330] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: stopping tablet replica
I20260620 23:08:50.750237 8648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 23:08:50.750330 9181 raft_consensus.cc:2229] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260620 23:08:50.750357 8648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:08:50.750380 9181 raft_consensus.cc:2258] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:50.750463 9179 ts_tablet_manager.cc:1905] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.750792 9181 ts_tablet_manager.cc:1905] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.751993 9179 ts_tablet_manager.cc:1918] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:50.752055 9179 log.cc:1196] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/1c205601b875483db94eef8ccf3f8ff2
I20260620 23:08:50.752229 9181 ts_tablet_manager.cc:1918] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:50.752269 9179 ts_tablet_manager.cc:1939] T 1c205601b875483db94eef8ccf3f8ff2 P 1afb552a650f457c99fac04f93ae69b0: Deleting consensus metadata
I20260620 23:08:50.752274 9181 log.cc:1196] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/1c205601b875483db94eef8ccf3f8ff2
I20260620 23:08:50.752295 8925 tablet_service.cc:1467] Processing CreateTablet for tablet d90f8b1e370543878674c82ea001be54 (DEFAULT_TABLE table=test-table [id=a72cb3c37dae4cc2be640475f255551d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.752434 8925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d90f8b1e370543878674c82ea001be54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.752492 9181 ts_tablet_manager.cc:1939] T 1c205601b875483db94eef8ccf3f8ff2 P e07000a737d940558ed7ef7b25f08da4: Deleting consensus metadata
I20260620 23:08:50.752692 9180 ts_tablet_manager.cc:1905] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:50.753551 9157 tablet_bootstrap.cc:492] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0: Bootstrap starting.
I20260620 23:08:50.753605 8633 catalog_manager.cc:4841] TS 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66:36713): tablet 1c205601b875483db94eef8ccf3f8ff2 (table test-table [id=648d9df330f8423dbc48fa754bfc8258]) successfully deleted
I20260620 23:08:50.753809 8632 catalog_manager.cc:4841] TS e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173): tablet 1c205601b875483db94eef8ccf3f8ff2 (table test-table [id=648d9df330f8423dbc48fa754bfc8258]) successfully deleted
I20260620 23:08:50.753959 9061 tablet_service.cc:1467] Processing CreateTablet for tablet d90f8b1e370543878674c82ea001be54 (DEFAULT_TABLE table=test-table [id=a72cb3c37dae4cc2be640475f255551d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.754042 9061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d90f8b1e370543878674c82ea001be54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.754191 9157 tablet_bootstrap.cc:654] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.754678 9157 tablet_bootstrap.cc:492] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0: No bootstrap required, opened a new log
I20260620 23:08:50.754705 9157 ts_tablet_manager.cc:1397] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.754799 9157 raft_consensus.cc:348] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.754837 9157 raft_consensus.cc:374] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.754850 9157 raft_consensus.cc:729] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afb552a650f457c99fac04f93ae69b0, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.754940 9157 consensus_queue.cc:260] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.755021 9157 ts_tablet_manager.cc:1428] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.755086 9156 tablet_bootstrap.cc:492] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: Bootstrap starting.
I20260620 23:08:50.755345 9156 tablet_bootstrap.cc:654] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.755458 8784 tablet_service.cc:1467] Processing CreateTablet for tablet d90f8b1e370543878674c82ea001be54 (DEFAULT_TABLE table=test-table [id=a72cb3c37dae4cc2be640475f255551d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:08:50.755810 9156 tablet_bootstrap.cc:492] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: No bootstrap required, opened a new log
I20260620 23:08:50.755843 9156 ts_tablet_manager.cc:1397] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 23:08:50.755915 9180 ts_tablet_manager.cc:1918] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:08:50.755926 8784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d90f8b1e370543878674c82ea001be54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:50.755972 9180 log.cc:1196] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/1c205601b875483db94eef8ccf3f8ff2
I20260620 23:08:50.755944 9156 raft_consensus.cc:348] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.755999 9156 raft_consensus.cc:374] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.756024 9156 raft_consensus.cc:729] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07000a737d940558ed7ef7b25f08da4, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.756065 9156 consensus_queue.cc:260] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.756134 9156 ts_tablet_manager.cc:1428] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:50.756173 9180 ts_tablet_manager.cc:1939] T 1c205601b875483db94eef8ccf3f8ff2 P 3f0a5708a012449ab80fa30286c7f82e: Deleting consensus metadata
I20260620 23:08:50.756448 8632 catalog_manager.cc:4841] TS 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029): tablet 1c205601b875483db94eef8ccf3f8ff2 (table test-table [id=648d9df330f8423dbc48fa754bfc8258]) successfully deleted
I20260620 23:08:50.757069 9162 tablet_bootstrap.cc:492] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e: Bootstrap starting.
I20260620 23:08:50.757495 9162 tablet_bootstrap.cc:654] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:50.758049 9162 tablet_bootstrap.cc:492] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e: No bootstrap required, opened a new log
I20260620 23:08:50.758111 9162 ts_tablet_manager.cc:1397] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:50.758232 9162 raft_consensus.cc:348] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.758286 9162 raft_consensus.cc:374] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:50.758299 9162 raft_consensus.cc:729] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0a5708a012449ab80fa30286c7f82e, State: Initialized, Role: FOLLOWER
I20260620 23:08:50.758352 9162 consensus_queue.cc:260] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:50.758440 9162 ts_tablet_manager.cc:1428] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:51.078090 9161 raft_consensus.cc:482] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:08:51.078189 9161 raft_consensus.cc:504] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:51.078497 9161 leader_election.cc:288] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:51.081945 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.078737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:39679 (local address 127.1.32.65:38029)
0620 23:08:51.078813 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:51.078815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:51.078933 (+ 118us) server_negotiation.cc:407] Connection header received
0620 23:08:51.079041 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:51.079042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:51.079054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:51.079110 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:51.079280 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.079781 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.081207 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.081545 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.081643 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.081655 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:51.081818 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 23:08:51.081868 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:08:51.081948 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.078662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:38029 (local address 127.1.32.66:39679)
0620 23:08:51.078830 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:51.078842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:51.078960 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:51.079129 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:51.079129 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:51.079132 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:51.079195 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.079197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.079854 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.079855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.081113 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.081116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.081651 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.081652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.081751 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.081755 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:08:51.081807 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:08:51.081856 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260620 23:08:51.081987 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.078984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:43079 (local address 127.1.32.67:39173)
0620 23:08:51.079099 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:51.079101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:51.079109 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:51.079131 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:51.079132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:51.079142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:51.079201 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:08:51.079392 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.079862 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.081228 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.081603 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.081682 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.081712 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:51.081867 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 23:08:51.081918 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260620 23:08:51.081951 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.078877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39173 (local address 127.1.32.66:43079)
0620 23:08:51.078999 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:51.079009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:51.079063 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:51.079244 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:51.079245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:51.079247 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:08:51.079292 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.079293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.079938 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.079939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.081113 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.081116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.081737 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.081738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.081801 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.081804 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:08:51.081861 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 23:08:51.081888 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":36,"thread_start_us":58,"threads_started":1}
I20260620 23:08:51.082242 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90f8b1e370543878674c82ea001be54" candidate_uuid: "1afb552a650f457c99fac04f93ae69b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0a5708a012449ab80fa30286c7f82e" is_pre_election: true
I20260620 23:08:51.082325 8805 raft_consensus.cc:2454] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1afb552a650f457c99fac04f93ae69b0 in term 0.
I20260620 23:08:51.082381 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90f8b1e370543878674c82ea001be54" candidate_uuid: "1afb552a650f457c99fac04f93ae69b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4" is_pre_election: true
I20260620 23:08:51.082463 9085 raft_consensus.cc:2454] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1afb552a650f457c99fac04f93ae69b0 in term 0.
I20260620 23:08:51.082499 8876 leader_election.cc:302] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1afb552a650f457c99fac04f93ae69b0, 3f0a5708a012449ab80fa30286c7f82e; no voters:
I20260620 23:08:51.082679 9161 raft_consensus.cc:2781] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:08:51.082720 9161 raft_consensus.cc:482] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:08:51.082739 9161 raft_consensus.cc:3037] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:51.083498 9161 raft_consensus.cc:504] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:51.083621 9161 leader_election.cc:288] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [CANDIDATE]: Term 1 election: Requested vote from peers 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029), e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173)
I20260620 23:08:51.083783 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90f8b1e370543878674c82ea001be54" candidate_uuid: "1afb552a650f457c99fac04f93ae69b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0a5708a012449ab80fa30286c7f82e"
I20260620 23:08:51.083882 8805 raft_consensus.cc:3037] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:51.083835 9085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d90f8b1e370543878674c82ea001be54" candidate_uuid: "1afb552a650f457c99fac04f93ae69b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e07000a737d940558ed7ef7b25f08da4"
I20260620 23:08:51.083946 9085 raft_consensus.cc:3037] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:51.084519 8805 raft_consensus.cc:2454] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1afb552a650f457c99fac04f93ae69b0 in term 1.
I20260620 23:08:51.084635 9085 raft_consensus.cc:2454] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1afb552a650f457c99fac04f93ae69b0 in term 1.
I20260620 23:08:51.084677 8876 leader_election.cc:302] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1afb552a650f457c99fac04f93ae69b0, 3f0a5708a012449ab80fa30286c7f82e; no voters:
I20260620 23:08:51.084774 9161 raft_consensus.cc:2781] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:51.084895 9161 raft_consensus.cc:686] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 1 LEADER]: Becoming Leader. State: Replica: 1afb552a650f457c99fac04f93ae69b0, State: Running, Role: LEADER
I20260620 23:08:51.084977 9161 consensus_queue.cc:237] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } }
I20260620 23:08:51.085790 8648 catalog_manager.cc:5495] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1afb552a650f457c99fac04f93ae69b0 (127.1.32.66). New cstate: current_term: 1 leader_uuid: "1afb552a650f457c99fac04f93ae69b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1afb552a650f457c99fac04f93ae69b0" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 36713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 } health_report { overall_health: UNKNOWN } } }
I20260620 23:08:51.111378 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.109010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39738 (local address 127.1.32.66:36713)
0620 23:08:51.109157 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:51.109159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:51.109189 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:08:51.109223 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:51.109223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:51.109243 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:51.109313 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:51.109433 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.109935 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.110594 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.110782 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.110832 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.111006 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:51.111151 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:51.111227 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:51.111282 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 23:08:51.111301 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260620 23:08:51.115635 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.113693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:44751 (local address 127.1.32.65:38029)
0620 23:08:51.113737 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:51.113739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:51.113750 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:51.113795 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:51.113795 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:51.113802 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:51.113845 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:51.113959 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.114369 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.115079 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.115253 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.115298 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.115464 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:51.115487 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:51.115525 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:51.115568 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 23:08:51.115582 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:08:51.115692 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.113549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:38029 (local address 127.1.32.66:44751)
0620 23:08:51.113696 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:51.113706 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:51.113753 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:51.113854 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:51.113855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:51.113856 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:51.113905 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.113906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.114441 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.114442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.115013 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.115015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.115324 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.115325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.115388 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.115399 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:51.115533 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:51.115535 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:08:51.115567 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 23:08:51.115601 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260620 23:08:51.117725 9085 raft_consensus.cc:1261] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Refusing update from remote peer 1afb552a650f457c99fac04f93ae69b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:51.117738 8805 raft_consensus.cc:1261] T d90f8b1e370543878674c82ea001be54 P 3f0a5708a012449ab80fa30286c7f82e [term 1 FOLLOWER]: Refusing update from remote peer 1afb552a650f457c99fac04f93ae69b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:08:51.117990 9161 consensus_queue.cc:1035] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e07000a737d940558ed7ef7b25f08da4" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:51.118114 9237 consensus_queue.cc:1035] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0a5708a012449ab80fa30286c7f82e" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:08:51.124760 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.122638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:52935 (local address 127.1.32.66:36713)
0620 23:08:51.122677 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:51.122679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:51.122706 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:08:51.122729 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:51.122730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:51.122737 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:51.122771 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:08:51.122889 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.123342 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.124006 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.124227 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.124285 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.124482 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:08:51.124516 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:08:51.124579 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:51.124668 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 23:08:51.124690 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:08:51.124787 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.122545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:36713 (local address 127.1.32.65:52935)
0620 23:08:51.122630 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:51.122640 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:51.122673 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:51.122775 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:51.122776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:51.122777 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 23:08:51.122807 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.122808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.123437 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.123438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.123941 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:51.123943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:51.124320 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:51.124321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:51.124402 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:51.124408 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 23:08:51.124588 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 23:08:51.124590 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:08:51.124653 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 23:08:51.124700 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260620 23:08:51.131146 8647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:51.131234 8647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:08:51.131809 8647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1513ac8b1b674e5ca355233fb9f0886e: Sending DeleteTablet for 3 replicas of tablet d90f8b1e370543878674c82ea001be54
I20260620 23:08:51.132017 8925 tablet_service.cc:1514] Processing DeleteTablet for tablet d90f8b1e370543878674c82ea001be54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:51 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:39726
I20260620 23:08:51.132112 9061 tablet_service.cc:1514] Processing DeleteTablet for tablet d90f8b1e370543878674c82ea001be54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:08:51 UTC) from {username='oryx', principal='oryx/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:46998
I20260620 23:08:51.132220 9179 tablet_replica.cc:330] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0: stopping tablet replica
I20260620 23:08:51.132272 9179 raft_consensus.cc:2229] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:08:51.132361 9179 raft_consensus.cc:2258] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:51.132431 9181 tablet_replica.cc:330] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: stopping tablet replica
I20260620 23:08:51.132483 9181 raft_consensus.cc:2229] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:08:51.132520 9181 raft_consensus.cc:2258] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:08:51.133044 9181 ts_tablet_manager.cc:1905] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:08:51.134544 9181 ts_tablet_manager.cc:1918] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 23:08:51.134601 9181 log.cc:1196] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/d90f8b1e370543878674c82ea001be54
I20260620 23:08:51.134799 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 8720
I20260620 23:08:51.134820 9181 ts_tablet_manager.cc:1939] T d90f8b1e370543878674c82ea001be54 P e07000a737d940558ed7ef7b25f08da4: Deleting consensus metadata
I20260620 23:08:51.138200 8632 catalog_manager.cc:4841] TS e07000a737d940558ed7ef7b25f08da4 (127.1.32.67:39173): tablet d90f8b1e370543878674c82ea001be54 (table test-table [id=a72cb3c37dae4cc2be640475f255551d]) successfully deleted
I20260620 23:08:51.142108 9179 ts_tablet_manager.cc:1905] T d90f8b1e370543878674c82ea001be54 P 1afb552a650f457c99fac04f93ae69b0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 23:08:51.142495 8632 connection.cc:535] client connection to 127.1.32.65:38029 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 23:08:51.142601 8632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:08:51.142822 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 8860
I20260620 23:08:51.143993 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.143811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:38029 (local address 127.0.0.1:52218)
0620 23:08:51.143904 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:51.143953 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":60,"threads_started":1}
W20260620 23:08:51.144164 8632 catalog_manager.cc:4584] TS 3f0a5708a012449ab80fa30286c7f82e (127.1.32.65:38029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d90f8b1e370543878674c82ea001be54: Network error: Client connection negotiation failed: client connection to 127.1.32.65:38029: connect: Connection refused (error 111)
I20260620 23:08:51.147995 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:51.147896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:36713 (local address 127.0.0.1:39744)
0620 23:08:51.147942 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:51.147970 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:36713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 23:08:51.148132 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 9000
I20260620 23:08:51.153282 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 8616
2026-06-20T23:08:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2869 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:51 dist-test-slave-fzjm krb5kdc[9251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:51 dist-test-slave-fzjm krb5kdc[9251](info): set up 2 sockets
Jun 20 23:08:51 dist-test-slave-fzjm krb5kdc[9251](info): commencing operation
krb5kdc: starting...
W20260620 23:08:52.159478 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:52 dist-test-slave-fzjm krb5kdc[9251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:52.267071 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43113
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:43113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:52.343243 9264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:52.343461 9264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:52.343499 9264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:52.344624 9264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:52.344684 9264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:52.344708 9264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:52.344729 9264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:52.344770 9264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:52.346230 9264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:43113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:52.346485 9264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:52.346720 9264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:52.351192 9272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:52.351183 9270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:52.351186 9269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:52.351436 9264 server_base.cc:1034] running on GCE node
I20260620 23:08:52.351643 9264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:52.351886 9264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:52.353041 9264 hybrid_clock.cc:648] HybridClock initialized: now 1781996932353024 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T23:08:52Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1199 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:08:52 dist-test-slave-fzjm krb5kdc[9277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:08:52 dist-test-slave-fzjm krb5kdc[9277](info): set up 2 sockets
Jun 20 23:08:52 dist-test-slave-fzjm krb5kdc[9277](info): commencing operation
krb5kdc: starting...
W20260620 23:08:53.355400 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:08:53 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:08:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:08:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:08:53.461910 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:41007
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:41007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:53.536715 9290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:53.536931 9290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:53.536968 9290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:53.538152 9290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:53.538215 9290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:53.538240 9290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:53.538269 9290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:53.538300 9290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:53.539728 9290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:41007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:41007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:53.539974 9290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:53.540181 9290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:08:53.544677 9290 server_base.cc:1034] running on GCE node
W20260620 23:08:53.544637 9295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:53.544648 9296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:53.544608 9298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:53.545034 9290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:53.545303 9290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:53.546460 9290 hybrid_clock.cc:648] HybridClock initialized: now 1781996933546448 us; error 31 us; skew 500 ppm
Jun 20 23:08:53 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:53.552982 9290 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:53.553403 9290 webserver.cc:460] Webserver started at http://127.1.32.126:46053/ using document root <none> and password file <none>
I20260620 23:08:53.553579 9290 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:53.553614 9290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:53.553706 9290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:53.554649 9290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "80d449316b5b42b0917e1c33d1d920af"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "3e41d5980f1eaa2d6d7b68b7757fd717"
server_key_iv: "0d1952e82b63d8ecaba4fb1744aa01cc"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.554999 9290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "80d449316b5b42b0917e1c33d1d920af"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "3e41d5980f1eaa2d6d7b68b7757fd717"
server_key_iv: "0d1952e82b63d8ecaba4fb1744aa01cc"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.556169 9290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:53.556818 9305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.556962 9290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.557037 9290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "80d449316b5b42b0917e1c33d1d920af"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "3e41d5980f1eaa2d6d7b68b7757fd717"
server_key_iv: "0d1952e82b63d8ecaba4fb1744aa01cc"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.557087 9290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:53.585711 9290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:53.588215 9290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:53.588366 9290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:53.591966 9290 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:41007
I20260620 23:08:53.592278 9357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:41007 every 8 connection(s)
I20260620 23:08:53.592336 9290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:53.592979 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:08:53.595430 9358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Bootstrap starting.
I20260620 23:08:53.595722 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 9290
I20260620 23:08:53.595818 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:08:53.596014 1153 external_mini_cluster.cc:1428] Setting key 146bffb2253480074751429d5f55fd3d
I20260620 23:08:53.596019 9358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Neither blocks nor log segments found. Creating new log.
I20260620 23:08:53.596364 9358 log.cc:824] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:53.597092 9358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: No bootstrap required, opened a new log
I20260620 23:08:53.598747 9358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:53.598879 9358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:08:53.598948 9358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d449316b5b42b0917e1c33d1d920af, State: Initialized, Role: FOLLOWER
I20260620 23:08:53.599109 9358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:53.599193 9358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:53.599234 9358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:53.599288 9358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:08:53.599745 9358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:53.599846 9358 leader_election.cc:302] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80d449316b5b42b0917e1c33d1d920af; no voters:
I20260620 23:08:53.600034 9358 leader_election.cc:288] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:08:53.600059 9363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:08:53.600231 9363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 LEADER]: Becoming Leader. State: Replica: 80d449316b5b42b0917e1c33d1d920af, State: Running, Role: LEADER
I20260620 23:08:53.600335 9358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:53.600368 9363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:53.600664 9364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80d449316b5b42b0917e1c33d1d920af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } } }
I20260620 23:08:53.600754 9364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:53.600652 9363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80d449316b5b42b0917e1c33d1d920af. Latest consensus state: current_term: 1 leader_uuid: "80d449316b5b42b0917e1c33d1d920af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } } }
I20260620 23:08:53.601025 9363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:53.601151 9370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:53.601774 9370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:08:53 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:53.603368 9370 catalog_manager.cc:1347] Generated new cluster ID: d8274a2915f644c8bc57e1722e26c9b9
I20260620 23:08:53.603421 9370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:53.606971 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:53.596487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59114 (local address 127.1.32.126:41007)
0620 23:08:53.596646 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:53.596653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:53.596677 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:08:53.597060 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:53.597065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:53.597106 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:53.597273 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:53.597861 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.598628 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.599558 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.599978 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.600084 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:53.602830 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:53.602834 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:53.602834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:53.602845 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:53.604967 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:53.605353 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:53.605355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:53.605356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:53.605446 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:53.605627 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:53.605628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:53.605628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:53.605897 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:53.606042 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:53.606489 (+ 447us) server_negotiation.cc:299] Negotiation successful
0620 23:08:53.606611 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260620 23:08:53.624570 9370 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:08:53.625231 9370 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:53.629688 9370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Generated new TSK 0
I20260620 23:08:53.629829 9370 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:08:53.659992 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41007
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:53.735294 9384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:53.735476 9384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:53.735497 9384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:53.736644 9384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:53.736683 9384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:53.736729 9384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:53.738082 9384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:53.738268 9384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:53.738473 9384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:53.743161 9389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:53.743224 9390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:53.743340 9384 server_base.cc:1034] running on GCE node
W20260620 23:08:53.743168 9392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:53.743656 9384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:53.743897 9384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:53.745044 9384 hybrid_clock.cc:648] HybridClock initialized: now 1781996933745036 us; error 32 us; skew 500 ppm
Jun 20 23:08:53 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:53.747366 9384 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:08:53.747754 9384 webserver.cc:460] Webserver started at http://127.1.32.65:35945/ using document root <none> and password file <none>
I20260620 23:08:53.747949 9384 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:53.747988 9384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:53.748070 9384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:53.748979 9384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "cd028083a54c4cada32002b43419d18c"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "7b9486baa790eab928f31427ce83b291"
server_key_iv: "876c030a45be5ffd6392851d01a42679"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.749259 9384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "cd028083a54c4cada32002b43419d18c"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "7b9486baa790eab928f31427ce83b291"
server_key_iv: "876c030a45be5ffd6392851d01a42679"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.750325 9384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:08:53.750937 9399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.751117 9384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.751223 9384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "cd028083a54c4cada32002b43419d18c"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "7b9486baa790eab928f31427ce83b291"
server_key_iv: "876c030a45be5ffd6392851d01a42679"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.751289 9384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:53.776364 9384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:53.778154 9384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:53.778301 9384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:53.778518 9384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:53.778811 9384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:53.778849 9384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.778887 9384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:53.778930 9384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.784564 9384 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:35701
I20260620 23:08:53.784610 9512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:35701 every 8 connection(s)
I20260620 23:08:53.784958 9384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:08:53 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:53.793535 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:53.785614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:44085 (local address 127.1.32.126:41007)
0620 23:08:53.785674 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:53.785676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:53.785863 (+ 187us) server_negotiation.cc:407] Connection header received
0620 23:08:53.786318 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:53.786319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:53.786349 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:53.786399 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:53.787362 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.787859 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.788787 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.788998 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.789052 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:53.791053 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:53.791056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:53.791056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:53.791064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:53.792226 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:53.792612 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:53.792613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:53.792614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:53.792670 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:53.792904 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:53.792905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:53.792905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:53.793013 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:53.793083 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:53.793356 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 23:08:53.793448 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:08:53.793678 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:53.785611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:44085)
0620 23:08:53.785746 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:53.785779 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:53.786242 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:53.786452 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:53.786455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:53.787041 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:53.787234 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:53.787239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.787938 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.787938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:53.788702 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:53.788703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.789064 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.789065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:53.789156 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:53.789586 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:53.789594 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:53.790969 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:53.792306 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:53.792314 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:53.792316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:53.792528 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:53.792731 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:53.792731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:53.792732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:53.792841 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:53.793073 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:53.793079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:53.793212 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 23:08:53.793311 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 23:08:53.793461 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260620 23:08:53.794126 9513 heartbeater.cc:347] Connected to a master server at 127.1.32.126:41007
I20260620 23:08:53.794219 9513 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:53.794355 9513 heartbeater.cc:511] Master 127.1.32.126:41007 requested a full tablet report, sending...
I20260620 23:08:53.794628 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 9384
I20260620 23:08:53.794741 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:08:53.794957 1153 external_mini_cluster.cc:1428] Setting key 51beac908dbac09302d93e0de4a998bb
I20260620 23:08:53.794970 9322 ts_manager.cc:194] Registered new tserver with Master: cd028083a54c4cada32002b43419d18c (127.1.32.65:35701)
I20260620 23:08:53.795589 9322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:44085
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:08:53.817579 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41007
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:53.895779 9518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:53.895982 9518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:53.896004 9518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:53.897115 9518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:53.897167 9518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:53.897193 9518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:53.898609 9518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:53.898852 9518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:53.899075 9518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:08:53.904060 9518 server_base.cc:1034] running on GCE node
W20260620 23:08:53.904096 9523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:53.904064 9524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:53.904223 9526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:53.904449 9518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:53.904695 9518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:53.905849 9518 hybrid_clock.cc:648] HybridClock initialized: now 1781996933905843 us; error 58 us; skew 500 ppm
Jun 20 23:08:53 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:53.908141 9518 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:08:53.908533 9518 webserver.cc:460] Webserver started at http://127.1.32.66:45627/ using document root <none> and password file <none>
I20260620 23:08:53.908700 9518 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:53.908736 9518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:53.908815 9518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:53.909785 9518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "eae83befcc074f2fb4695ca1ed173342"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "d80d72cffe8b310427775b797140de07"
server_key_iv: "e0bac7ebbbdb95baefa53fd5fd9ca783"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.910092 9518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "eae83befcc074f2fb4695ca1ed173342"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "d80d72cffe8b310427775b797140de07"
server_key_iv: "e0bac7ebbbdb95baefa53fd5fd9ca783"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.911335 9518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:08:53.911993 9533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.912156 9518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:08:53.912237 9518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "eae83befcc074f2fb4695ca1ed173342"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "d80d72cffe8b310427775b797140de07"
server_key_iv: "e0bac7ebbbdb95baefa53fd5fd9ca783"
server_key_version: "encryptionkey@0"
I20260620 23:08:53.912290 9518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:53.928784 9518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:53.930881 9518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:53.931022 9518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:53.931176 9518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:53.931394 9518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:53.931421 9518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.931440 9518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:53.931452 9518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:53.936757 9518 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:45205
I20260620 23:08:53.936857 9646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:45205 every 8 connection(s)
I20260620 23:08:53.937127 9518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:53.942019 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 9518
I20260620 23:08:53.942106 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:08:53.942291 1153 external_mini_cluster.cc:1428] Setting key f22758e5d4a11b2e0d5d71535b6af42d
Jun 20 23:08:53 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:53.946132 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:53.937839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:35731 (local address 127.1.32.126:41007)
0620 23:08:53.937896 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:53.937897 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:53.938120 (+ 223us) server_negotiation.cc:407] Connection header received
0620 23:08:53.938551 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:53.938552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:53.938562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:53.938614 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:53.939524 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.940088 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.941037 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.941207 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.941262 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:53.943484 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:53.943485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:53.943486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:53.943493 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:53.944797 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:53.945230 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:53.945231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:53.945231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:53.945286 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:53.945527 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:53.945528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:53.945528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:53.945629 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:53.945707 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:53.945969 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 23:08:53.946047 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:08:53.946302 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:53.937779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:35731)
0620 23:08:53.937981 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:53.938014 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:53.938460 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:53.938666 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:53.938668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:53.939255 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:53.939437 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:53.939442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.940174 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.940175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:53.940958 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:53.940960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:53.941280 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:53.941281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:53.941419 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:53.941850 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:53.941858 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:53.943395 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:53.944902 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:53.944908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:53.944910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:53.945161 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:53.945351 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:53.945352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:53.945352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:53.945460 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:53.945688 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:53.945706 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:53.945827 (+ 121us) client_negotiation.cc:769] Sending connection context
0620 23:08:53.945947 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 23:08:53.946092 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260620 23:08:53.946728 9647 heartbeater.cc:347] Connected to a master server at 127.1.32.126:41007
I20260620 23:08:53.946812 9647 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:53.947014 9647 heartbeater.cc:511] Master 127.1.32.126:41007 requested a full tablet report, sending...
I20260620 23:08:53.947398 9322 ts_manager.cc:194] Registered new tserver with Master: eae83befcc074f2fb4695ca1ed173342 (127.1.32.66:45205)
I20260620 23:08:53.947758 9322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:35731
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:08:53.965700 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41007
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:54.042527 9652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:54.042712 9652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:54.042732 9652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:54.043768 9652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:54.043799 9652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:54.043828 9652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:54.045508 9652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:54.045774 9652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:54.046037 9652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:54.051096 9657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.051071 9660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.051090 9658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:54.051548 9652 server_base.cc:1034] running on GCE node
I20260620 23:08:54.051764 9652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:54.052006 9652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:54.053161 9652 hybrid_clock.cc:648] HybridClock initialized: now 1781996934053150 us; error 37 us; skew 500 ppm
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996934, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:54.060860 9652 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:08:54.061305 9652 webserver.cc:460] Webserver started at http://127.1.32.67:38025/ using document root <none> and password file <none>
I20260620 23:08:54.061539 9652 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:54.061602 9652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:54.061740 9652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:08:54.062799 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "acf952cfa51e48adb011274fcf8ab250"
format_stamp: "Formatted at 2026-06-20 23:08:54 on dist-test-slave-fzjm"
server_key: "2d5452f7cdb9475daa8113452e19125a"
server_key_iv: "b997db0cdd196b3b4550682161e8a33b"
server_key_version: "encryptionkey@0"
I20260620 23:08:54.063223 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "acf952cfa51e48adb011274fcf8ab250"
format_stamp: "Formatted at 2026-06-20 23:08:54 on dist-test-slave-fzjm"
server_key: "2d5452f7cdb9475daa8113452e19125a"
server_key_iv: "b997db0cdd196b3b4550682161e8a33b"
server_key_version: "encryptionkey@0"
I20260620 23:08:54.064512 9652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:54.065196 9667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.065362 9652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:54.065464 9652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "acf952cfa51e48adb011274fcf8ab250"
format_stamp: "Formatted at 2026-06-20 23:08:54 on dist-test-slave-fzjm"
server_key: "2d5452f7cdb9475daa8113452e19125a"
server_key_iv: "b997db0cdd196b3b4550682161e8a33b"
server_key_version: "encryptionkey@0"
I20260620 23:08:54.065532 9652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:54.078851 9652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:54.080695 9652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:54.080847 9652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:54.081053 9652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:54.081355 9652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:54.081406 9652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.081437 9652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:54.081465 9652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.087320 9652 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:40363
I20260620 23:08:54.087397 9780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:40363 every 8 connection(s)
I20260620 23:08:54.087689 9652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:08:54.090027 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 9652
I20260620 23:08:54.090112 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:08:54.090291 1153 external_mini_cluster.cc:1428] Setting key 077e78dde7936d7780ab396f04333870
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996934, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:08:54.096922 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.088437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:54297 (local address 127.1.32.126:41007)
0620 23:08:54.088508 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.088511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.088680 (+ 169us) server_negotiation.cc:407] Connection header received
0620 23:08:54.089242 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.089244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.089257 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.089313 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.090314 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.091111 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.092070 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.092272 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.092340 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.094321 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:54.094322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:54.094323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:54.094330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:54.095595 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:54.095985 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:54.095986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:54.095986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:54.096040 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:54.096287 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:54.096287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:54.096288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:54.096391 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:54.096460 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:54.096732 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 23:08:54.096834 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:08:54.097110 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.088389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:54297)
0620 23:08:54.088568 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.088607 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.089123 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.089414 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.089418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.089967 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.090217 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.090222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.091191 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.091192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.091979 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.091981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.092356 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.092357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.092504 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.092940 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.092947 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.094231 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:08:54.095683 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:54.095690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:54.095692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:54.095905 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:54.096123 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:08:54.096124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:08:54.096124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:08:54.096220 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:08:54.096451 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:08:54.096457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:08:54.096604 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 23:08:54.096700 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:08:54.096850 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260620 23:08:54.097565 9781 heartbeater.cc:347] Connected to a master server at 127.1.32.126:41007
I20260620 23:08:54.097666 9781 heartbeater.cc:464] Registering TS with master...
I20260620 23:08:54.097797 9781 heartbeater.cc:511] Master 127.1.32.126:41007 requested a full tablet report, sending...
I20260620 23:08:54.098112 9322 ts_manager.cc:194] Registered new tserver with Master: acf952cfa51e48adb011274fcf8ab250 (127.1.32.67:40363)
I20260620 23:08:54.098428 9322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:54297
I20260620 23:08:54.101702 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab.
Entry for principal oryx/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab.
WARNING: no policy specified for oryx/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab.
Entry for principal oryx/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab.
WARNING: no policy specified for oryx/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab.
Entry for principal oryx/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab.
I20260620 23:08:54.163599 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 9384
I20260620 23:08:54.167824 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 9518
I20260620 23:08:54.171932 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 9652
I20260620 23:08:54.176213 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 9290
I20260620 23:08:54.181344 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:41007
--webserver_interface=127.1.32.126
--webserver_port=46053
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:41007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:08:54.276887 9790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:54.277189 9790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:54.277226 9790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:54.279335 9790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:08:54.279390 9790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:54.279415 9790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:54.279445 9790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:08:54.279484 9790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:08:54.281934 9790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:41007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:41007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=46053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:54.282126 9790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:54.282289 9790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:54.286963 9795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.287002 9798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.287191 9796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:54.287165 9790 server_base.cc:1034] running on GCE node
I20260620 23:08:54.287405 9790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:54.287664 9790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:54.288796 9790 hybrid_clock.cc:648] HybridClock initialized: now 1781996934288784 us; error 29 us; skew 500 ppm
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996934, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:54.291316 9790 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:08:54.291775 9790 webserver.cc:460] Webserver started at http://127.1.32.126:46053/ using document root <none> and password file <none>
I20260620 23:08:54.292002 9790 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:54.292098 9790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:54.293417 9790 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:54.294093 9805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.294268 9790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:54.294373 9790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "80d449316b5b42b0917e1c33d1d920af"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "3e41d5980f1eaa2d6d7b68b7757fd717"
server_key_iv: "0d1952e82b63d8ecaba4fb1744aa01cc"
server_key_version: "encryptionkey@0"
I20260620 23:08:54.294710 9790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:54.304045 9790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:54.306177 9790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:54.306326 9790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:54.310225 9790 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:41007
I20260620 23:08:54.310258 9857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:41007 every 8 connection(s)
I20260620 23:08:54.310626 9790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:08:54.312158 9858 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:08:54.312804 9858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Bootstrap starting.
I20260620 23:08:54.316570 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 9790
I20260620 23:08:54.316848 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:35701
--local_ip_for_outbound_sockets=127.1.32.65
--tserver_master_addrs=127.1.32.126:41007
--webserver_port=35945
--webserver_interface=127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:08:54.322185 9858 log.cc:824] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Log is configured to *not* fsync() on all Append() calls
I20260620 23:08:54.324615 9858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:08:54.324962 9858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Bootstrap complete.
I20260620 23:08:54.329246 9858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:54.329571 9858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d449316b5b42b0917e1c33d1d920af, State: Initialized, Role: FOLLOWER
I20260620 23:08:54.329730 9858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:54.329802 9858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:08:54.329842 9858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:08:54.329891 9858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:08:54.331276 9858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:54.331400 9858 leader_election.cc:302] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80d449316b5b42b0917e1c33d1d920af; no voters:
I20260620 23:08:54.331565 9858 leader_election.cc:288] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:08:54.331632 9862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:08:54.331812 9862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [term 2 LEADER]: Becoming Leader. State: Replica: 80d449316b5b42b0917e1c33d1d920af, State: Running, Role: LEADER
I20260620 23:08:54.331941 9862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } }
I20260620 23:08:54.331815 9858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:08:54.332196 9864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80d449316b5b42b0917e1c33d1d920af. Latest consensus state: current_term: 2 leader_uuid: "80d449316b5b42b0917e1c33d1d920af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } } }
I20260620 23:08:54.332417 9864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:54.332201 9863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "80d449316b5b42b0917e1c33d1d920af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d449316b5b42b0917e1c33d1d920af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41007 } } }
I20260620 23:08:54.332666 9863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af [sys.catalog]: This master's current role is: LEADER
I20260620 23:08:54.332808 9869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:08:54.333278 9869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:08:54.333446 9869 catalog_manager.cc:1259] Loaded cluster ID: d8274a2915f644c8bc57e1722e26c9b9
I20260620 23:08:54.333550 9869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:08:54.335086 9869 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:08:54.335264 9869 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 80d449316b5b42b0917e1c33d1d920af: Loaded TSK: 0
I20260620 23:08:54.335493 9869 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:08:54.405062 9860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:54.405237 9860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:54.405259 9860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:54.406406 9860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:54.406445 9860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:54.406469 9860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:08:54.407878 9860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.65.keytab
--principal=oryx/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:35701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=35945
--tserver_master_addrs=127.1.32.126:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:54.408074 9860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:54.408289 9860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:54.412801 9884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:54.412922 9860 server_base.cc:1034] running on GCE node
W20260620 23:08:54.412812 9887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.413017 9885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:54.413213 9860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:54.413494 9860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:54.414669 9860 hybrid_clock.cc:648] HybridClock initialized: now 1781996934414660 us; error 42 us; skew 500 ppm
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996934, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:54.417503 9860 init.cc:377] Logged in from keytab as oryx/127.1.32.65@KRBTEST.COM (short username oryx)
I20260620 23:08:54.417930 9860 webserver.cc:460] Webserver started at http://127.1.32.65:35945/ using document root <none> and password file <none>
I20260620 23:08:54.418176 9860 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:54.418262 9860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:54.419894 9860 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:54.420503 9894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.420666 9860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:08:54.420763 9860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "cd028083a54c4cada32002b43419d18c"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "7b9486baa790eab928f31427ce83b291"
server_key_iv: "876c030a45be5ffd6392851d01a42679"
server_key_version: "encryptionkey@0"
I20260620 23:08:54.421077 9860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:54.438143 9860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:54.440395 9860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:54.440544 9860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:54.440764 9860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:54.441041 9860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:54.441089 9860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.441126 9860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:54.441151 9860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.447512 9860 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:35701
I20260620 23:08:54.447593 10007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:35701 every 8 connection(s)
I20260620 23:08:54.447897 9860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:08:54.452292 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 9860
I20260620 23:08:54.452559 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:45205
--local_ip_for_outbound_sockets=127.1.32.66
--tserver_master_addrs=127.1.32.126:41007
--webserver_port=45627
--webserver_interface=127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:08:54.459470 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.448770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:33637 (local address 127.1.32.126:41007)
0620 23:08:54.451021 (+ 2251us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.451030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.451074 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:08:54.451187 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.451193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.451243 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.451362 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.452318 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.453686 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.455189 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.455673 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.455772 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.459330 (+ 3558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:33637: BlockingRecv error: recv got EOF from 127.1.32.65:33637 (error 108)
Metrics: {"server-negotiator.queue_time_us":2202,"thread_start_us":74,"threads_started":1}
I20260620 23:08:54.459476 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.448651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:33637)
0620 23:08:54.448857 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.448914 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.449534 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.451423 (+ 1889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.451427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.451939 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.452201 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.452207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.453797 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.453798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.455082 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.455084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.455767 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.455768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.455959 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.456502 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.456513 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.459231 (+ 2718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":113,"threads_started":1}
W20260620 23:08:54.459710 10008 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:41007 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.32.126:41007: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:54.464511 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.459930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:43635)
0620 23:08:54.459994 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.460005 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.460043 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.460324 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.460325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.460429 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.460468 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.460469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.461420 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.461421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.461877 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.461878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.462352 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.462353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.462395 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.462805 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.462810 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.464446 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 23:08:54.464658 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.459989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:43635 (local address 127.1.32.126:41007)
0620 23:08:54.460193 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.460196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.460205 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:54.460227 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.460228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.460241 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.460321 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.460541 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.461347 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.461949 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.462279 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.462356 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.464518 (+ 2162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:43635: BlockingRecv error: recv got EOF from 127.1.32.65:43635 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":103,"threads_started":1}
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:54.468937 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.464721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:48261)
0620 23:08:54.464767 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.464834 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.464875 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.465008 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.465009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.465105 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.465128 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.465129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.465987 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.465987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.466449 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.466450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.466937 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.466938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.466987 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.467340 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.467345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.468853 (+ 1508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 23:08:54.468952 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.464804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:48261 (local address 127.1.32.126:41007)
0620 23:08:54.464848 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.464850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.464861 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:54.464942 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.464943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.464952 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.464996 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.465184 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.465903 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.466518 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.466823 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.466906 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.468915 (+ 2009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:48261: BlockingRecv error: recv got EOF from 127.1.32.65:48261 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 23:08:54.469033 10008 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:08:54.534153 10012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:54.534338 10012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:54.534360 10012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:54.535461 10012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:54.535496 10012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:54.535542 10012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:08:54.536963 10012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.66.keytab
--principal=oryx/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:45205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=45627
--tserver_master_addrs=127.1.32.126:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:54.537154 10012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:54.537354 10012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:54.541911 10018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.541880 10021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:54.542086 10012 server_base.cc:1034] running on GCE node
W20260620 23:08:54.541893 10019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:54.542325 10012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:54.542552 10012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:54.543720 10012 hybrid_clock.cc:648] HybridClock initialized: now 1781996934543714 us; error 31 us; skew 500 ppm
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996934, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:54.546060 10012 init.cc:377] Logged in from keytab as oryx/127.1.32.66@KRBTEST.COM (short username oryx)
I20260620 23:08:54.546454 10012 webserver.cc:460] Webserver started at http://127.1.32.66:45627/ using document root <none> and password file <none>
I20260620 23:08:54.546674 10012 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:54.546751 10012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:54.547971 10012 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:54.548429 10028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.548580 10012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.548660 10012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "eae83befcc074f2fb4695ca1ed173342"
format_stamp: "Formatted at 2026-06-20 23:08:53 on dist-test-slave-fzjm"
server_key: "d80d72cffe8b310427775b797140de07"
server_key_iv: "e0bac7ebbbdb95baefa53fd5fd9ca783"
server_key_version: "encryptionkey@0"
I20260620 23:08:54.548944 10012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:54.564642 10012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:54.566272 10012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:54.566417 10012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:54.566628 10012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:54.566977 10012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:54.567035 10012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.567070 10012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:54.567097 10012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.574111 10012 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:45205
I20260620 23:08:54.574213 10141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:45205 every 8 connection(s)
I20260620 23:08:54.574532 10012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:08:54.577174 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10012
I20260620 23:08:54.577435 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:40363
--local_ip_for_outbound_sockets=127.1.32.67
--tserver_master_addrs=127.1.32.126:41007
--webserver_port=38025
--webserver_interface=127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 23:08:54.581952 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.575270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:54847 (local address 127.1.32.126:41007)
0620 23:08:54.575329 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.575331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.575547 (+ 216us) server_negotiation.cc:407] Connection header received
0620 23:08:54.576040 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.576042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.576057 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.576118 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.577002 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.577598 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.578649 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.578844 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.578960 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.581900 (+ 2940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:54847: BlockingRecv error: recv got EOF from 127.1.32.66:54847 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:08:54.582000 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.575292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:54847)
0620 23:08:54.575429 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.575462 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.575945 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.576187 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.576192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.576727 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.576903 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.576908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.577700 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.577701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.578548 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.578552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.578975 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.578976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.579121 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.579604 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.579613 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.581824 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
W20260620 23:08:54.582163 10142 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:41007 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.32.126:41007: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:54.586148 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.582355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:56953)
0620 23:08:54.582424 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.582433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.582475 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.582590 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.582591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.582701 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.582737 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.582738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.583355 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.583356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.583799 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.583800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.584185 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.584186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.584250 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.584632 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.584636 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.586076 (+ 1440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260620 23:08:54.586197 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.582392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:56953 (local address 127.1.32.126:41007)
0620 23:08:54.582424 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.582426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.582462 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:08:54.582527 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.582527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.582537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.582586 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.582811 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.583280 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.583909 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.584110 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.584184 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.586139 (+ 1955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:56953: BlockingRecv error: recv got EOF from 127.1.32.66:56953 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:54.590176 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.586568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:33073)
0620 23:08:54.586644 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.586654 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.586696 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.586814 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.586814 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.586972 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.586999 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.587000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.587556 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.587558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.588009 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.588010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.588359 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.588360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.588404 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.588741 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.588746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.590109 (+ 1363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260620 23:08:54.590222 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.586613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:33073 (local address 127.1.32.126:41007)
0620 23:08:54.586644 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.586646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.586683 (+ 37us) server_negotiation.cc:407] Connection header received
0620 23:08:54.586750 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.586751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.586759 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.586810 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.587072 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.587490 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.588095 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.588282 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.588340 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.590169 (+ 1829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:33073: BlockingRecv error: recv got EOF from 127.1.32.66:33073 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260620 23:08:54.590456 10142 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:08:54.655723 10145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:08:54.655905 10145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:08:54.655927 10145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:08:54.657018 10145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:08:54.657051 10145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:08:54.657076 10145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:08:54.658289 10145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/krb5kdc/oryx_127.1.32.67.keytab
--principal=oryx/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:40363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=38025
--tserver_master_addrs=127.1.32.126:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:08:54.658442 10145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:08:54.658593 10145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:08:54.663130 10150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.663129 10151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:08:54.663156 10153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:08:54.663154 10145 server_base.cc:1034] running on GCE node
I20260620 23:08:54.663501 10145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:08:54.663739 10145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:08:54.664893 10145 hybrid_clock.cc:648] HybridClock initialized: now 1781996934664883 us; error 23 us; skew 500 ppm
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996934, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:08:54.667160 10145 init.cc:377] Logged in from keytab as oryx/127.1.32.67@KRBTEST.COM (short username oryx)
I20260620 23:08:54.667552 10145 webserver.cc:460] Webserver started at http://127.1.32.67:38025/ using document root <none> and password file <none>
I20260620 23:08:54.667773 10145 fs_manager.cc:359] Metadata directory not provided
I20260620 23:08:54.667842 10145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:08:54.669018 10145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:08:54.669552 10160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.669716 10145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.669802 10145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "acf952cfa51e48adb011274fcf8ab250"
format_stamp: "Formatted at 2026-06-20 23:08:54 on dist-test-slave-fzjm"
server_key: "2d5452f7cdb9475daa8113452e19125a"
server_key_iv: "b997db0cdd196b3b4550682161e8a33b"
server_key_version: "encryptionkey@0"
I20260620 23:08:54.670089 10145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:08:54.696892 10145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:08:54.698709 10145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:08:54.698853 10145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:08:54.699096 10145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:08:54.699375 10145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:08:54.699421 10145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.699452 10145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:08:54.699492 10145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:08:54.704939 10145 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:40363
I20260620 23:08:54.704996 10273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:40363 every 8 connection(s)
I20260620 23:08:54.705291 10145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:54.712145 10276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.705911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:55893)
0620 23:08:54.706097 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.706131 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.706617 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.706858 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.706862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.707475 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.707654 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.707659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.708329 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.708330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.709123 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.709125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.709513 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.709514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.709638 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.710049 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.710056 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.712012 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
W20260620 23:08:54.712131 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.705934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:55893 (local address 127.1.32.126:41007)
0620 23:08:54.706006 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.706007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.706211 (+ 204us) server_negotiation.cc:407] Connection header received
0620 23:08:54.706716 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.706718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.706729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.706782 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.707755 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.708232 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.709212 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.709406 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.709506 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.712085 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:55893: BlockingRecv error: recv got EOF from 127.1.32.67:55893 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:08:54.712320 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10145
W20260620 23:08:54.712492 10274 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:41007 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.32.126:41007: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:54.717515 10276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.712765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:55623)
0620 23:08:54.712807 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.712819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.712864 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.713032 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.713033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.713208 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.713261 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.713262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.713920 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.713921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.714576 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.714578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.715018 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.715019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.715094 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.715529 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.715533 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.717472 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260620 23:08:54.717655 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.712843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:55623 (local address 127.1.32.126:41007)
0620 23:08:54.712901 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.712903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.712910 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:54.712929 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.712930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.712942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.713025 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.713329 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.713848 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.714677 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.714875 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.715019 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.717580 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:55623: BlockingRecv error: recv got EOF from 127.1.32.67:55623 (error 108)
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":44}
I20260620 23:08:54.720124 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.712842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59120 (local address 127.1.32.126:41007)
0620 23:08:54.712899 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.712901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.712937 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:08:54.713023 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.713023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.713030 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.713084 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.713714 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.714198 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.715168 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.715452 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.715527 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.716818 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:08:54.716823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:08:54.716824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:08:54.716838 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:08:54.718766 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:54.719131 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:54.719135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:54.719135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:54.719227 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:08:54.719433 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:08:54.719433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:08:54.719434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:08:54.719652 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:08:54.719806 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 23:08:54.719913 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 23:08:54.720034 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:54 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:54.722049 10276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:54.717944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:59329)
0620 23:08:54.717988 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:54.717999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:54.718039 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:54.718152 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:54.718153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:54.718278 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:54.718305 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.718307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.718866 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.718868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.719359 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:54.719361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.719749 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.719750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:54.719806 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.720222 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:54.720227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:54.721970 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260620 23:08:54.722102 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:54.717999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:59329 (local address 127.1.32.126:41007)
0620 23:08:54.718051 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:54.718053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:54.718061 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:54.718082 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:54.718083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:54.718097 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:54.718140 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:54.718369 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.718774 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.719474 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:54.719666 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:54.719747 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:54.722040 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:59329: BlockingRecv error: recv got EOF from 127.1.32.67:59329 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260620 23:08:54.722229 10274 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 23:08:55 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:55 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:55.474301 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:55.469427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:57101)
0620 23:08:55.469560 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:55.469572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:55.469640 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:55.469781 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:55.469781 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:55.469943 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:55.469990 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:55.469991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.470662 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.470663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:55.471271 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:55.471273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.471690 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.471691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:55.471735 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:55.472288 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:55.472294 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:55.474225 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
W20260620 23:08:55.474397 10279 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:55.469489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57101 (local address 127.1.32.126:41007)
0620 23:08:55.469659 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:55.469661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:55.469669 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:55.469693 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:55.469693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:55.469703 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:55.469768 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:55.470060 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.470574 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.471417 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.471613 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.471678 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:55.474312 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:57101: BlockingRecv error: recv got EOF from 127.1.32.65:57101 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
Jun 20 23:08:55 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:55 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:55.595484 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:55.590803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:45665)
0620 23:08:55.590933 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:55.590956 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:55.591032 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:55.591186 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:55.591187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:55.591358 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:55.591402 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:55.591404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.592053 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.592054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:55.592663 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:55.592665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.593120 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.593121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:55.593170 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:55.593656 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:55.593660 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:55.595414 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
W20260620 23:08:55.595631 10279 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:55.590926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:45665 (local address 127.1.32.126:41007)
0620 23:08:55.591011 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:55.591013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:55.591024 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:08:55.591093 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:55.591094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:55.591112 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:55.591182 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:55.591478 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.591976 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.592765 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.593005 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.593061 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:55.595552 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:45665: BlockingRecv error: recv got EOF from 127.1.32.66:45665 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 23:08:55 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:55 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:55.727106 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:55.722592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:57377)
0620 23:08:55.722730 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:55.722740 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:55.722804 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:55.722945 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:55.722946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:55.723147 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:55.723193 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:55.723195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.723864 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.723864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:55.724492 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:55.724495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.724863 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.724864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:55.724924 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:55.725376 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:55.725380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:55.727042 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
W20260620 23:08:55.727228 10279 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:55.722639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:57377 (local address 127.1.32.126:41007)
0620 23:08:55.722704 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:55.722706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:55.722802 (+ 96us) server_negotiation.cc:407] Connection header received
0620 23:08:55.722855 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:55.722856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:55.722866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:55.722935 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:55.723250 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.723775 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.724601 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:55.724785 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:55.724856 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:55.727131 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:57377: BlockingRecv error: recv got EOF from 127.1.32.67:57377 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 23:08:56 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:56 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:56.479445 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:56.474867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:34889)
0620 23:08:56.475015 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:56.475025 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:56.475071 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:56.475278 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:56.475279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:56.475526 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:56.475588 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:56.475591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.476267 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.476268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:56.476793 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:56.476795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.477142 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.477143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:56.477179 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:56.477621 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:56.477625 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:56.479345 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
W20260620 23:08:56.479595 10283 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:56.474973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:34889 (local address 127.1.32.126:41007)
0620 23:08:56.475101 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:56.475103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:56.475113 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:08:56.475139 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:56.475140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:56.475157 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:56.475255 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:56.475670 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.476192 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.476869 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.477076 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.477139 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:56.479471 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:34889: BlockingRecv error: recv got EOF from 127.1.32.65:34889 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
Jun 20 23:08:56 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:56 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:56.600117 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:56.595906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:52619)
0620 23:08:56.596034 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:56.596043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:56.596100 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:56.596215 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:56.596216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:56.596396 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:56.596444 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:56.596445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.597125 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.597126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:56.597651 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:56.597652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.597949 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.597949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:56.597986 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:56.598461 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:56.598465 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:56.600058 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
W20260620 23:08:56.600272 10283 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:56.595978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:52619 (local address 127.1.32.126:41007)
0620 23:08:56.596042 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:56.596044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:56.596096 (+ 52us) server_negotiation.cc:407] Connection header received
0620 23:08:56.596141 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:56.596142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:56.596152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:56.596202 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:56.596500 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.597046 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.597706 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.597893 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.597943 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:56.600182 (+ 2239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:52619: BlockingRecv error: recv got EOF from 127.1.32.66:52619 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 23:08:56 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:56 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:56.731417 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:56.727512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:33727)
0620 23:08:56.727650 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:56.727661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:56.727700 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:56.727817 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:56.727818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:56.727987 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:56.728045 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:56.728046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.728608 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.728609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:56.729074 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:56.729075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.729355 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.729356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:56.729389 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:56.729840 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:56.729844 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:56.731360 (+ 1516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
W20260620 23:08:56.731539 10283 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:56.727619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:33727 (local address 127.1.32.126:41007)
0620 23:08:56.727679 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:56.727681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:56.727688 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:08:56.727750 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:56.727751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:56.727761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:56.727803 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:56.728098 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.728551 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.729134 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:56.729303 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:56.729348 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:56.731435 (+ 2087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:33727: BlockingRecv error: recv got EOF from 127.1.32.67:33727 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 23:08:57 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:57 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:57.484381 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:57.479868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:57551)
0620 23:08:57.480014 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:57.480023 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:57.480069 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:57.480190 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:57.480191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:57.480347 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:57.480394 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:57.480396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.481020 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.481020 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:57.481542 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:57.481545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.481922 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.481923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:57.481959 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:57.482428 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:57.482433 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:57.484315 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
W20260620 23:08:57.484516 10287 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:57.479940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57551 (local address 127.1.32.126:41007)
0620 23:08:57.480073 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:57.480075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:57.480084 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:08:57.480105 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:57.480106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:57.480116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:57.480176 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:57.480453 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.480954 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.481634 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.481854 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.481922 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:57.484408 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:57551: BlockingRecv error: recv got EOF from 127.1.32.65:57551 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
Jun 20 23:08:57 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:57 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:57.604763 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:57.600531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:45153)
0620 23:08:57.600710 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:57.600721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:57.600793 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:57.600926 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:57.600927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:57.601083 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:57.601125 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:57.601127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.601719 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.601720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:57.602201 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:57.602202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.602514 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.602515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:57.602566 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:57.602993 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:57.602997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:57.604665 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
W20260620 23:08:57.604811 10287 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:57.600614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:45153 (local address 127.1.32.126:41007)
0620 23:08:57.600694 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:57.600696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:57.600777 (+ 81us) server_negotiation.cc:407] Connection header received
0620 23:08:57.600854 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:57.600855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:57.600868 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:57.600921 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:57.601193 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.601648 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.602272 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.602444 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.602513 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:57.604747 (+ 2234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:45153: BlockingRecv error: recv got EOF from 127.1.32.66:45153 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 20 23:08:57 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:57 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:57.736239 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:57.731826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:37515)
0620 23:08:57.731982 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:57.731993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:57.732071 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:57.732229 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:57.732230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:57.732393 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:57.732436 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:57.732438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.733005 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.733007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:57.733548 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:57.733550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.733899 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.733900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:57.733957 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:57.734415 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:57.734425 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:57.736108 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
W20260620 23:08:57.736284 10287 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:57.731896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:37515 (local address 127.1.32.126:41007)
0620 23:08:57.731970 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:57.731973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:57.732052 (+ 79us) server_negotiation.cc:407] Connection header received
0620 23:08:57.732122 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:57.732123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:57.732139 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:57.732223 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:57.732513 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.732938 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.733640 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:57.733815 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:57.733897 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:57.736194 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:37515: BlockingRecv error: recv got EOF from 127.1.32.67:37515 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 20 23:08:58 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:58 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:58.489475 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:58.484832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:58951)
0620 23:08:58.484963 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:58.484973 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:58.485020 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:58.485167 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:58.485167 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:58.485320 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:58.485365 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:58.485367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.485960 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.485961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:58.486489 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:58.486491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.486819 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.486820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:58.486857 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:58.487404 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:58.487410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:58.489418 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
W20260620 23:08:58.489609 10291 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:58.484913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:58951 (local address 127.1.32.126:41007)
0620 23:08:58.485036 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:58.485038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:58.485052 (+ 14us) server_negotiation.cc:407] Connection header received
0620 23:08:58.485072 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:58.485073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:58.485082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:58.485139 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:58.485444 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.485901 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.486566 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.486753 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.486809 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:58.489498 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:58951: BlockingRecv error: recv got EOF from 127.1.32.65:58951 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
Jun 20 23:08:58 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:58 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:58.612496 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:58.605282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:46357)
0620 23:08:58.605474 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:58.605497 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:58.605561 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:58.605679 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:58.605679 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:58.605856 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:58.605902 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:58.605904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.606559 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.606560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:58.607199 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:58.607202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.607535 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.607535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:58.607573 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:58.610594 (+ 3021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:58.610599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:58.612441 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":85,"threads_started":1}
W20260620 23:08:58.612644 10291 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:58.605348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:46357 (local address 127.1.32.126:41007)
0620 23:08:58.605419 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:58.605421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:58.605567 (+ 146us) server_negotiation.cc:407] Connection header received
0620 23:08:58.605622 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:58.605622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:58.605633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:58.605669 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:58.605978 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.606497 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.607283 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.607476 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.607527 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:58.612560 (+ 5033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:46357: BlockingRecv error: recv got EOF from 127.1.32.66:46357 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 23:08:58 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:58 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:58.740938 10293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:58.736713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:34511)
0620 23:08:58.736861 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:58.736872 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:58.736935 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:58.737060 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:58.737060 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:58.737211 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:58.737258 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:58.737259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.737882 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.737882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:58.738374 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:58.738375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.738686 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.738687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:58.738721 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:58.739202 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:58.739207 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:58.740881 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
W20260620 23:08:58.741096 10291 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:58.736821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:34511 (local address 127.1.32.126:41007)
0620 23:08:58.736885 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:58.736887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:58.736918 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:08:58.736997 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:58.736998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:58.737011 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:58.737056 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:58.737314 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.737817 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.738449 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:58.738623 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:58.738678 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:58.741012 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:34511: BlockingRecv error: recv got EOF from 127.1.32.67:34511 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 20 23:08:59 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:59 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:59.494297 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:59.489907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:40363)
0620 23:08:59.490071 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:59.490083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:59.490143 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:59.490264 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:59.490265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:59.490437 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:59.490484 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:59.490485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.491239 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.491240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:59.491782 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:59.491783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.492104 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.492105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:59.492142 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:59.492572 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:59.492576 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:59.494230 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
W20260620 23:08:59.494423 10295 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:59.489991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:40363 (local address 127.1.32.126:41007)
0620 23:08:59.490090 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:59.490092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:59.490128 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:08:59.490183 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:59.490184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:59.490194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:59.490248 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:59.490567 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.491170 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.491851 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.492042 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.492095 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:59.494321 (+ 2226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:40363: BlockingRecv error: recv got EOF from 127.1.32.65:40363 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":53,"threads_started":1}
Jun 20 23:08:59 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:59 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:59.617784 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:59.612951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:42845)
0620 23:08:59.613126 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:59.613154 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:59.613212 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:59.613456 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:59.613457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:59.613636 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:59.613687 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:59.613688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.614380 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.614382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:59.614978 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:59.614979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.615352 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.615354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:59.615419 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:59.615912 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:59.615924 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:59.617717 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
W20260620 23:08:59.617918 10295 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:59.613107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:42845 (local address 127.1.32.126:41007)
0620 23:08:59.613183 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:59.613185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:59.613193 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:08:59.613325 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:59.613326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:59.613338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:59.613448 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:59.613781 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.614294 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.615055 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.615265 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.615351 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:59.617839 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:42845: BlockingRecv error: recv got EOF from 127.1.32.66:42845 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 23:08:59 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:08:59 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:08:59.746181 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:08:59.741458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:60331)
0620 23:08:59.741621 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:08:59.741633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:08:59.741715 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:08:59.741871 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:08:59.741872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:08:59.742047 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:08:59.742091 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:59.742093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.742732 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.742733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:59.743329 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:08:59.743330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.743707 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.743708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:08:59.743762 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:59.744236 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:08:59.744241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:08:59.746082 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
W20260620 23:08:59.746368 10295 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:08:59.741557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:60331 (local address 127.1.32.126:41007)
0620 23:08:59.741620 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:08:59.741622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:08:59.741697 (+ 75us) server_negotiation.cc:407] Connection header received
0620 23:08:59.741781 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:08:59.741782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:08:59.741795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:08:59.741866 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:08:59.742174 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.742657 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.743405 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:08:59.743627 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:08:59.743705 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:08:59.746217 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:60331: BlockingRecv error: recv got EOF from 127.1.32.67:60331 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 23:09:00 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:00 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:00.499421 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:00.494732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:35215)
0620 23:09:00.494864 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:00.494876 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:00.494965 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:00.495128 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:00.495129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:00.495278 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:00.495326 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:00.495327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.495945 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.495945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:00.496450 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:00.496451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.496800 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.496801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:00.496837 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:00.497350 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:00.497354 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:00.499362 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":50,"threads_started":1}
W20260620 23:09:00.499581 10299 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:00.494817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:35215 (local address 127.1.32.126:41007)
0620 23:09:00.495005 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:00.495007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:00.495015 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:09:00.495037 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:00.495037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:00.495049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:00.495114 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:00.495388 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.495881 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.496522 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.496733 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.496793 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:00.499472 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:35215: BlockingRecv error: recv got EOF from 127.1.32.65:35215 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
Jun 20 23:09:00 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:00 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:00.622735 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:00.618259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:33147)
0620 23:09:00.618417 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:00.618428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:00.618503 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:00.618652 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:00.618653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:00.618825 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:00.618874 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:00.618876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.619544 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.619545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:00.620079 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:00.620080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.620413 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.620414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:00.620455 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:00.620928 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:00.620933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:00.622622 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
W20260620 23:09:00.622778 10299 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:00.618372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:33147 (local address 127.1.32.126:41007)
0620 23:09:00.618417 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:00.618420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:00.618485 (+ 65us) server_negotiation.cc:407] Connection header received
0620 23:09:00.618562 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:00.618563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:00.618574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:00.618629 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:00.618971 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.619445 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.620156 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.620346 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.620398 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:00.622722 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:33147: BlockingRecv error: recv got EOF from 127.1.32.66:33147 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 20 23:09:00 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:00 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:00.751111 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:00.746626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:54205)
0620 23:09:00.746752 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:00.746764 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:00.746835 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:00.746969 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:00.746970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:00.747168 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:00.747212 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:00.747214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.747937 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.747938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:00.748443 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:00.748444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.748755 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.748756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:00.748792 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:00.749209 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:00.749212 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:00.751055 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
W20260620 23:09:00.751225 10299 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:00.746672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:54205 (local address 127.1.32.126:41007)
0620 23:09:00.746734 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:00.746736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:00.746834 (+ 98us) server_negotiation.cc:407] Connection header received
0620 23:09:00.746873 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:00.746874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:00.746905 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:00.746953 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:00.747294 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.747879 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.748514 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:00.748698 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:00.748743 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:00.751135 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:54205: BlockingRecv error: recv got EOF from 127.1.32.67:54205 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 23:09:01 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:01 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:01.504025 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:01.499875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:50073)
0620 23:09:01.500027 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:01.500036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:01.500090 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:01.500212 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:01.500213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:01.500355 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:01.500400 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:01.500401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.501032 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.501032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:01.501545 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:01.501546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.501863 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.501864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:01.501898 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:01.502354 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:01.502358 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:01.503943 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
W20260620 23:09:01.504194 10303 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:01.499947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:50073 (local address 127.1.32.126:41007)
0620 23:09:01.500049 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:01.500050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:01.500092 (+ 42us) server_negotiation.cc:407] Connection header received
0620 23:09:01.500130 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:01.500130 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:01.500140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:01.500200 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:01.500491 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.500952 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.501608 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.501809 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.501855 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:01.504098 (+ 2243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:50073: BlockingRecv error: recv got EOF from 127.1.32.65:50073 (error 108)
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":49,"threads_started":1}
Jun 20 23:09:01 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:01 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:01.627967 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:01.623428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:49097)
0620 23:09:01.623535 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:01.623546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:01.623605 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:01.623750 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:01.623750 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:01.623890 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:01.623976 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:01.623977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.624700 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.624700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:01.625254 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:01.625255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.625586 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.625587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:01.625623 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:01.626056 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:01.626061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:01.627911 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
W20260620 23:09:01.628075 10303 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:01.623454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:49097 (local address 127.1.32.126:41007)
0620 23:09:01.623523 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:01.623525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:01.623603 (+ 78us) server_negotiation.cc:407] Connection header received
0620 23:09:01.623665 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:01.623665 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:01.623676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:01.623727 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:01.624063 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.624632 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.625320 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.625521 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.625581 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:01.628002 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:49097: BlockingRecv error: recv got EOF from 127.1.32.66:49097 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 20 23:09:01 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:01 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:01.756253 10305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:01.751620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:37989)
0620 23:09:01.751802 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:01.751813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:01.751874 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:01.751994 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:01.751995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:01.752199 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:01.752253 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:01.752255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.752852 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.752853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:01.753367 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:01.753369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.753647 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.753648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:01.753684 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:01.754158 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:01.754162 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:01.756195 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
W20260620 23:09:01.756356 10303 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:01.751666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:37989 (local address 127.1.32.126:41007)
0620 23:09:01.751726 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:01.751728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:01.751871 (+ 143us) server_negotiation.cc:407] Connection header received
0620 23:09:01.751920 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:01.751921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:01.751933 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:01.751979 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:01.752310 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.752769 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.753433 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:01.753592 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:01.753635 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:01.756306 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:37989: BlockingRecv error: recv got EOF from 127.1.32.67:37989 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 23:09:02 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:02 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:02.508612 10306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:02.504418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:43267)
0620 23:09:02.504554 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:02.504563 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:02.504608 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:02.504772 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:02.504773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:02.504918 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:02.504965 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:02.504966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.505566 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.505566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:02.506068 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:02.506070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.506392 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.506393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:02.506434 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:02.506863 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:02.506867 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:02.508542 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
W20260620 23:09:02.508754 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:02.504525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:43267 (local address 127.1.32.126:41007)
0620 23:09:02.504646 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:02.504649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:02.504656 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:09:02.504674 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:02.504675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:02.504684 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:02.504751 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:02.505026 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.505501 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.506134 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.506328 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.506379 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:02.508637 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:43267: BlockingRecv error: recv got EOF from 127.1.32.65:43267 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
Jun 20 23:09:02 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:02 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:02.633347 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:02.628417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:34417)
0620 23:09:02.628536 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:02.628549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:02.628607 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:02.628810 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:02.628811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:02.629006 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:02.629052 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:02.629053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.629851 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.629853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:02.630420 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:02.630422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.630786 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.630787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:02.630850 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:02.631366 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:02.631370 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:02.633258 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
W20260620 23:09:02.633404 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:02.628519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:34417 (local address 127.1.32.126:41007)
0620 23:09:02.628583 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:02.628585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:02.628594 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:09:02.628702 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:02.628703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:02.628726 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:02.628803 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:02.629130 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.629755 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.630509 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.630713 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.630787 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:02.633328 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:34417: BlockingRecv error: recv got EOF from 127.1.32.66:34417 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 20 23:09:02 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:02 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:02.761303 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:02.756815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:55535)
0620 23:09:02.756966 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:02.756978 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:02.757067 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:02.757257 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:02.757258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:02.757433 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:02.757480 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:02.757482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.758159 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.758161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:02.758662 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:02.758664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.759080 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.759081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:02.759145 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:02.759540 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:02.759544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:02.761200 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
W20260620 23:09:02.761346 10307 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:02.756882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:55535 (local address 127.1.32.126:41007)
0620 23:09:02.756954 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:02.756957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:02.757048 (+ 91us) server_negotiation.cc:407] Connection header received
0620 23:09:02.757135 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:02.757136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:02.757150 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:02.757249 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:02.757566 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.758077 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.758758 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:02.759001 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:02.759069 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:02.761284 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:55535: BlockingRecv error: recv got EOF from 127.1.32.67:55535 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 20 23:09:03 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:03 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:03.513604 10310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:03.509107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:36169)
0620 23:09:03.509254 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:03.509268 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:03.509344 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:03.509477 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:03.509478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:03.509643 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:03.509703 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:03.509705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.510353 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.510353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:03.510915 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:03.510917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.511307 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.511308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:03.511358 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:03.511816 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:03.511820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:03.513546 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
W20260620 23:09:03.513734 10311 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:03.509155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:36169 (local address 127.1.32.126:41007)
0620 23:09:03.509325 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:03.509327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:03.509334 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:09:03.509391 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:03.509391 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:03.509402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:03.509455 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:03.509768 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.510285 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.510994 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.511193 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.511272 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:03.513647 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:36169: BlockingRecv error: recv got EOF from 127.1.32.65:36169 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
Jun 20 23:09:03 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:03 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:03.642139 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:03.633808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:45261)
0620 23:09:03.633915 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:03.633925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:03.633989 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:03.634091 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:03.634091 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:03.634247 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:03.634291 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:03.634293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.634951 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.634953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:03.635566 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:03.635568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.635972 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.635973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:03.636031 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:03.640017 (+ 3986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:03.640022 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:03.642029 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
W20260620 23:09:03.642238 10311 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:03.633867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:45261 (local address 127.1.32.126:41007)
0620 23:09:03.633945 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:03.633947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:03.633988 (+ 41us) server_negotiation.cc:407] Connection header received
0620 23:09:03.634031 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:03.634031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:03.634042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:03.634086 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:03.634348 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.634824 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.635685 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.635888 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.635972 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:03.642133 (+ 6161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:45261: BlockingRecv error: recv got EOF from 127.1.32.66:45261 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 20 23:09:03 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:03 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:03.766566 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:03.761725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:53483)
0620 23:09:03.761908 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:03.761921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:03.761998 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:03.762132 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:03.762133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:03.762307 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:03.762351 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:03.762353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.763072 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.763073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:03.763841 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:03.763844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.764217 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.764218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:03.764272 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:03.764772 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:03.764777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:03.766470 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
W20260620 23:09:03.766611 10311 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:03.761821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:53483 (local address 127.1.32.126:41007)
0620 23:09:03.761912 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:03.761915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:03.761980 (+ 65us) server_negotiation.cc:407] Connection header received
0620 23:09:03.762047 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:03.762048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:03.762061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:03.762125 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:03.762443 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.762997 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.763950 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:03.764145 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:03.764218 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:03.766550 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:53483: BlockingRecv error: recv got EOF from 127.1.32.67:53483 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 20 23:09:04 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:04 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:04.522704 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:04.514076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:37817)
0620 23:09:04.514219 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:04.514229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:04.514286 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:04.514457 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:04.514459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:04.514727 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:04.514787 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:04.514789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.515546 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.515548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:04.516045 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:04.516046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.516408 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.516409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:04.516465 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:04.517026 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:04.517031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:04.522613 (+ 5582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
W20260620 23:09:04.523104 10315 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:04.514137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:37817 (local address 127.1.32.126:41007)
0620 23:09:04.514273 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:04.514275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:04.514283 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:09:04.514359 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:04.514359 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:04.514371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:04.514448 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:04.514876 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.515463 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.516135 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.516329 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.516406 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:04.523037 (+ 6631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:37817: BlockingRecv error: recv got EOF from 127.1.32.65:37817 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
Jun 20 23:09:04 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:04 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:04.646996 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:04.642653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:56207)
0620 23:09:04.642813 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:04.642831 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:04.642937 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:04.643087 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:04.643088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:04.643262 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:04.643309 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:04.643311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.643937 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.643937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:04.644459 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:04.644460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.644764 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.644765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:04.644802 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:04.645223 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:04.645227 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:04.646833 (+ 1606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
W20260620 23:09:04.647125 10315 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:04.642730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:56207 (local address 127.1.32.126:41007)
0620 23:09:04.642779 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:04.642781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:04.642927 (+ 146us) server_negotiation.cc:407] Connection header received
0620 23:09:04.642996 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:04.642997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:04.643008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:04.643077 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:04.643376 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.643854 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.644524 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.644707 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.644754 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:04.647030 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:56207: BlockingRecv error: recv got EOF from 127.1.32.66:56207 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 23:09:04 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:04 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:04.771863 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:04.767153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:39841)
0620 23:09:04.767329 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:04.767341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:04.767429 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:04.767600 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:04.767601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:04.767766 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:04.767809 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:04.767811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.768449 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.768451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:04.768962 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:04.768963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.769307 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.769308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:04.769369 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:04.769840 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:04.769845 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:04.771804 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260620 23:09:04.771973 10315 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:04.767281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:39841 (local address 127.1.32.126:41007)
0620 23:09:04.767347 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:04.767349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:04.767411 (+ 62us) server_negotiation.cc:407] Connection header received
0620 23:09:04.767508 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:04.767509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:04.767523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:04.767581 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:04.767885 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.768362 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.769040 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:04.769228 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:04.769307 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:04.771881 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:39841: BlockingRecv error: recv got EOF from 127.1.32.67:39841 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 23:09:05 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:05 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:05.527736 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:05.523292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:51801)
0620 23:09:05.523403 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:05.523413 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:05.523467 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:05.523582 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:05.523582 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:05.523767 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:05.523829 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:05.523830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.524515 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.524515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:05.525036 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:05.525037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.525383 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.525383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:05.525419 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:05.525918 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:05.525923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:05.527668 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
W20260620 23:09:05.527858 10319 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:05.523335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:51801 (local address 127.1.32.126:41007)
0620 23:09:05.523448 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:05.523450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:05.523460 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:09:05.523509 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:05.523510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:05.523519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:05.523578 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:05.523926 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.524443 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.525106 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.525319 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.525377 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:05.527770 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:51801: BlockingRecv error: recv got EOF from 127.1.32.65:51801 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
Jun 20 23:09:05 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:05 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:05.652541 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:05.647489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:58105)
0620 23:09:05.647696 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:05.647705 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:05.647782 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:05.647953 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:05.647954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:05.648168 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:05.648227 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:05.648229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.648897 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.648899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:05.649429 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:05.649430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.649793 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.649794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:05.649858 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:05.650310 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:05.650314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:05.652456 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
W20260620 23:09:05.652601 10319 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:05.647546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:58105 (local address 127.1.32.126:41007)
0620 23:09:05.647630 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:05.647632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:05.647767 (+ 135us) server_negotiation.cc:407] Connection header received
0620 23:09:05.647856 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:05.647857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:05.647871 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:05.647948 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:05.648313 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.648810 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.649525 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.649713 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.649792 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:05.652558 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:58105: BlockingRecv error: recv got EOF from 127.1.32.66:58105 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 20 23:09:05 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:05 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:05.776559 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:05.772368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:55829)
0620 23:09:05.772498 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:05.772515 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:05.772566 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:05.772669 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:05.772670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:05.772826 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:05.772870 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:05.772872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.773495 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.773496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:05.774023 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:05.774025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.774315 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.774316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:05.774351 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:05.774797 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:05.774806 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:05.776494 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
W20260620 23:09:05.776685 10319 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:05.772418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:55829 (local address 127.1.32.126:41007)
0620 23:09:05.772458 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:05.772459 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:05.772552 (+ 93us) server_negotiation.cc:407] Connection header received
0620 23:09:05.772603 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:05.772604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:05.772613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:05.772653 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:05.772921 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.773435 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.774089 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:05.774260 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:05.774304 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:05.776576 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:55829: BlockingRecv error: recv got EOF from 127.1.32.67:55829 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Jun 20 23:09:06 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:06 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:06.533075 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:06.528248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:60027)
0620 23:09:06.528432 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:06.528451 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:06.528511 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:06.528643 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:06.528644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:06.528816 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:06.528861 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:06.528863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.529510 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.529511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:06.530018 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:06.530019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.530364 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.530365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:06.530404 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:06.530987 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:06.530993 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:06.532985 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":112,"threads_started":1}
W20260620 23:09:06.533146 10323 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:06.528357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:60027 (local address 127.1.32.126:41007)
0620 23:09:06.528467 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:06.528468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:06.528497 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:09:06.528564 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:06.528565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:06.528575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:06.528631 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:06.528937 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.529432 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.530093 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.530298 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.530357 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:06.533094 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:60027: BlockingRecv error: recv got EOF from 127.1.32.65:60027 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":49,"threads_started":1}
Jun 20 23:09:06 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:06 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:06.656975 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:06.652938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:56313)
0620 23:09:06.653058 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:06.653067 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:06.653108 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:06.653236 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:06.653236 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:06.653383 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:06.653429 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:06.653430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.653987 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.653988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:06.654443 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:06.654444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.654753 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.654754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:06.654793 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:06.655340 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:06.655344 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:06.656914 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
W20260620 23:09:06.657092 10323 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:06.653025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:56313 (local address 127.1.32.126:41007)
0620 23:09:06.653080 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:06.653088 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:06.653103 (+ 15us) server_negotiation.cc:407] Connection header received
0620 23:09:06.653155 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:06.653156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:06.653165 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:06.653224 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:06.653488 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.653927 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.654515 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.654688 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.654742 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:06.657011 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:56313: BlockingRecv error: recv got EOF from 127.1.32.66:56313 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 20 23:09:06 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:06 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:06.781726 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:06.777137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:52181)
0620 23:09:06.777264 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:06.777278 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:06.777353 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:06.777511 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:06.777512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:06.777743 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:06.777787 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:06.777789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.778423 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.778424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:06.779005 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:06.779007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.779342 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.779344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:06.779393 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:06.779893 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:06.779898 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:06.781645 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
W20260620 23:09:06.781781 10323 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:06.777177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:52181 (local address 127.1.32.126:41007)
0620 23:09:06.777221 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:06.777223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:06.777370 (+ 147us) server_negotiation.cc:407] Connection header received
0620 23:09:06.777416 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:06.777417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:06.777432 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:06.777487 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:06.777858 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.778359 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.779078 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:06.779259 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:06.779309 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:06.781736 (+ 2427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:52181: BlockingRecv error: recv got EOF from 127.1.32.67:52181 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Jun 20 23:09:07 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:07 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:07.538031 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:07.533549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:37175)
0620 23:09:07.533662 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:07.533674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:07.533741 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:07.533910 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:07.533910 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:07.534083 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:07.534129 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:07.534131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.534782 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.534783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:07.535403 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:07.535405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.535766 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.535767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:07.535808 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:07.536262 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:07.536266 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:07.537961 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
W20260620 23:09:07.538128 10327 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:07.533619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:37175 (local address 127.1.32.126:41007)
0620 23:09:07.533759 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:07.533761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:07.533769 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:09:07.533791 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:07.533791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:07.533815 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:07.533893 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:07.534218 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.534698 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.535497 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.535700 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.535756 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:07.538054 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:37175: BlockingRecv error: recv got EOF from 127.1.32.65:37175 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
Jun 20 23:09:07 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:07 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:07.661867 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:07.657495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:41887)
0620 23:09:07.657602 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:07.657619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:07.657690 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:07.657850 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:07.657851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:07.658070 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:07.658113 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:07.658115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.658725 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.658726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:07.659286 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:07.659288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.659600 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.659601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:07.659638 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:07.660076 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:07.660079 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:07.661805 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
W20260620 23:09:07.662009 10327 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:07.657615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:41887 (local address 127.1.32.126:41007)
0620 23:09:07.657685 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:07.657686 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:07.657695 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:09:07.657758 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:07.657759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:07.657771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:07.657829 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:07.658179 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.658648 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.659349 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.659523 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.659590 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:07.661922 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:41887: BlockingRecv error: recv got EOF from 127.1.32.66:41887 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 23:09:07 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:07 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:07.786827 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:07.782271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:40989)
0620 23:09:07.782477 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:07.782492 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:07.782560 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:07.782713 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:07.782714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:07.782858 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:07.782950 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:07.782952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.783534 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.783534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:07.784075 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:07.784076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.784408 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.784409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:07.784446 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:07.784877 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:07.784881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:07.786759 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
W20260620 23:09:07.786929 10327 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:07.782441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:40989 (local address 127.1.32.126:41007)
0620 23:09:07.782510 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:07.782512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:07.782546 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:09:07.782614 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:07.782615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:07.782649 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:07.782695 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:07.783012 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.783472 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.784169 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:07.784349 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:07.784403 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:07.786838 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:40989: BlockingRecv error: recv got EOF from 127.1.32.67:40989 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 20 23:09:08 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:08 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:08.542872 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:08.538541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:57113)
0620 23:09:08.538702 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:08.538713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:08.538762 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:08.538945 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:08.538945 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:08.539107 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:08.539153 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:08.539154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.539809 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.539810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:08.540312 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:08.540314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.540614 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.540614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:08.540650 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:08.541159 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:08.541164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:08.542814 (+ 1650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
W20260620 23:09:08.543069 10331 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:08.538645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57113 (local address 127.1.32.126:41007)
0620 23:09:08.538785 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:08.538787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:08.538795 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:09:08.538816 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:08.538817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:08.538827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:08.538952 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:08.539227 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.539729 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.540380 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.540555 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.540608 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:08.542971 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:57113: BlockingRecv error: recv got EOF from 127.1.32.65:57113 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
Jun 20 23:09:08 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:08 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:08.669490 10332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:08.662346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:49133)
0620 23:09:08.662505 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:08.662533 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:08.662611 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:08.662740 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:08.662741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:08.662973 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:08.663032 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:08.663034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.663650 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.663651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:08.664191 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:08.664193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.664485 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.664486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:08.664523 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:08.667696 (+ 3173us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:08.667700 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:08.669417 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
W20260620 23:09:08.669615 10331 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:08.662452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:49133 (local address 127.1.32.126:41007)
0620 23:09:08.662502 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:08.662503 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:08.662610 (+ 107us) server_negotiation.cc:407] Connection header received
0620 23:09:08.662669 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:08.662669 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:08.662680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:08.662725 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:08.663094 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.663564 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.664262 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.664427 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.664471 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:08.669527 (+ 5056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:49133: BlockingRecv error: recv got EOF from 127.1.32.66:49133 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 20 23:09:08 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:08 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.67@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:08.791658 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:08.787276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.67:32779)
0620 23:09:08.787437 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:08.787452 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:08.787510 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:08.787608 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:08.787608 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:08.787771 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:08.787815 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:08.787816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.788450 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.788450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:08.788960 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:08.788962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.789259 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.789260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:08.789300 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:08.789811 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:08.789816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:08.791585 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
W20260620 23:09:08.791721 10331 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:08.787342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:32779 (local address 127.1.32.126:41007)
0620 23:09:08.787386 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:08.787388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:08.787515 (+ 127us) server_negotiation.cc:407] Connection header received
0620 23:09:08.787554 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:08.787555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:08.787566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:08.787603 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:08.787873 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.788379 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.789029 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:08.789198 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:08.789245 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:08.791673 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.67:32779: BlockingRecv error: recv got EOF from 127.1.32.67:32779 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 23:09:09 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:09 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.65@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:09.548164 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:09.543429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.65:46973)
0620 23:09:09.543589 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:09.543600 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:09.543684 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:09.543912 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:09.543913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:09.544076 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:09.544122 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:09.544123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.544765 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.544766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:09.545508 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:09.545511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.545840 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.545841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:09.545877 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:09.546342 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:09.546347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:09.548098 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
W20260620 23:09:09.548257 10335 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:09.543505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:46973 (local address 127.1.32.126:41007)
0620 23:09:09.543625 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:09.543627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:09.543667 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:09:09.543727 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:09.543727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:09.543736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:09.543804 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:09.544207 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.544693 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.545562 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.545779 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.545835 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:09.548183 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.65:46973: BlockingRecv error: recv got EOF from 127.1.32.65:46973 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
Jun 20 23:09:09 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:09:09 dist-test-slave-fzjm krb5kdc[9277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.32.66@KRBTEST.COM for oryx/127.1.32.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:09:09.674434 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:09.670050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41007 (local address 127.1.32.66:42423)
0620 23:09:09.670210 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:09.670218 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:09.670266 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:09.670380 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:09.670381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:09.670529 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:09.670575 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:09.670577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.671196 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.671197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:09.671633 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:09.671634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.671934 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.671935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:09.671967 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:09.672452 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:09.672456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:09.674336 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.32.126:41007: Server oryx/127.1.32.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":67,"threads_started":1}
W20260620 23:09:09.674647 10335 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:09.670105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:42423 (local address 127.1.32.126:41007)
0620 23:09:09.670170 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:09.670172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:09.670268 (+ 96us) server_negotiation.cc:407] Connection header received
0620 23:09:09.670305 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:09.670306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:09.670315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:09.670363 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:09.670636 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.671136 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.671689 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.671869 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.671926 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:09.674441 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.32.66:42423: BlockingRecv error: recv got EOF from 127.1.32.66:42423 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:09:09.716956 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 9860
I20260620 23:09:09.721163 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10012
I20260620 23:09:09.724932 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10145
I20260620 23:09:09.728909 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 9790
2026-06-20T23:09:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17374 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T23:09:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:09:09Z Disabled control of system clock
I20260620 23:09:09.755600 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:41621
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:41621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 23:09:09.833091 10340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:09.833292 10340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:09.833312 10340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:09.834359 10340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:09:09.834393 10340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:09.834404 10340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:09.834416 10340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:09:09.834429 10340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:09:09.835662 10340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:41621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:41621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:09.835843 10340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:09.836063 10340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:09.841024 10346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:09.841061 10348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:09.841027 10345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:09.841120 10340 server_base.cc:1034] running on GCE node
I20260620 23:09:09.841513 10340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:09.841747 10340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:09.842911 10340 hybrid_clock.cc:648] HybridClock initialized: now 1781996949842863 us; error 38 us; skew 500 ppm
I20260620 23:09:09.844076 10340 webserver.cc:460] Webserver started at http://127.1.32.126:34833/ using document root <none> and password file <none>
I20260620 23:09:09.844288 10340 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:09.844329 10340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:09.844430 10340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:09.845439 10340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "136c7bd7cc464a69a5fa916cf045bf19"
format_stamp: "Formatted at 2026-06-20 23:09:09 on dist-test-slave-fzjm"
server_key: "647c7548a427660e10279423ff437a18"
server_key_iv: "cb0bf6e0d0603e0d7afe9812dcebc7c6"
server_key_version: "encryptionkey@0"
I20260620 23:09:09.845752 10340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "136c7bd7cc464a69a5fa916cf045bf19"
format_stamp: "Formatted at 2026-06-20 23:09:09 on dist-test-slave-fzjm"
server_key: "647c7548a427660e10279423ff437a18"
server_key_iv: "cb0bf6e0d0603e0d7afe9812dcebc7c6"
server_key_version: "encryptionkey@0"
I20260620 23:09:09.847031 10340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:09:09.847721 10354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:09.847904 10340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:09:09.848002 10340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "136c7bd7cc464a69a5fa916cf045bf19"
format_stamp: "Formatted at 2026-06-20 23:09:09 on dist-test-slave-fzjm"
server_key: "647c7548a427660e10279423ff437a18"
server_key_iv: "cb0bf6e0d0603e0d7afe9812dcebc7c6"
server_key_version: "encryptionkey@0"
I20260620 23:09:09.848074 10340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:09.860548 10340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:09.860831 10340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:09.860944 10340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:09.864737 10340 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:41621
I20260620 23:09:09.864768 10406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:41621 every 8 connection(s)
I20260620 23:09:09.865095 10340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:09:09.865707 10407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:09.868225 10407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19: Bootstrap starting.
I20260620 23:09:09.868795 10407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:09.869073 10407 log.cc:824] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:09.869546 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10340
I20260620 23:09:09.869640 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:09:09.869798 10407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19: No bootstrap required, opened a new log
I20260620 23:09:09.869804 1153 external_mini_cluster.cc:1428] Setting key 4e565f628e0d4c243a0dbe09d5695032
I20260620 23:09:09.872324 10407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c7bd7cc464a69a5fa916cf045bf19" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41621 } }
I20260620 23:09:09.872507 10407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:09.872542 10407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 136c7bd7cc464a69a5fa916cf045bf19, State: Initialized, Role: FOLLOWER
I20260620 23:09:09.872668 10407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c7bd7cc464a69a5fa916cf045bf19" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41621 } }
I20260620 23:09:09.872752 10407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:09:09.872809 10407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:09:09.872854 10407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:09.873524 10407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c7bd7cc464a69a5fa916cf045bf19" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41621 } }
I20260620 23:09:09.873628 10407 leader_election.cc:302] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 136c7bd7cc464a69a5fa916cf045bf19; no voters:
I20260620 23:09:09.873836 10407 leader_election.cc:288] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:09:09.873857 10412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:09:09.873992 10412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [term 1 LEADER]: Becoming Leader. State: Replica: 136c7bd7cc464a69a5fa916cf045bf19, State: Running, Role: LEADER
I20260620 23:09:09.874091 10407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:09:09.874126 10412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c7bd7cc464a69a5fa916cf045bf19" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41621 } }
I20260620 23:09:09.874507 10412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 136c7bd7cc464a69a5fa916cf045bf19. Latest consensus state: current_term: 1 leader_uuid: "136c7bd7cc464a69a5fa916cf045bf19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c7bd7cc464a69a5fa916cf045bf19" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41621 } } }
I20260620 23:09:09.874539 10414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "136c7bd7cc464a69a5fa916cf045bf19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136c7bd7cc464a69a5fa916cf045bf19" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 41621 } } }
I20260620 23:09:09.874667 10412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [sys.catalog]: This master's current role is: LEADER
I20260620 23:09:09.874679 10414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19 [sys.catalog]: This master's current role is: LEADER
I20260620 23:09:09.874922 10421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:09:09.875705 10421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:09:09.875963 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:09.870330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.1.32.126:41621)
0620 23:09:09.870552 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:09.870560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:09.870583 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:09:09.871084 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:09.871091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:09.871149 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:09.871256 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:09.871479 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.872312 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.873439 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:09.874032 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:09.874959 (+ 927us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:09.875109 (+ 150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:09.875112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:09.875112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:09.875301 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:09.875345 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:09.875359 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:09.875448 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:09.875542 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 23:09:09.875726 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260620 23:09:09.877327 10421 catalog_manager.cc:1347] Generated new cluster ID: c5cd5d228ec242e1b5948187be68da34
I20260620 23:09:09.877377 10421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:09:09.882949 10421 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:09:09.883463 10421 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:09:09.889063 10421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 136c7bd7cc464a69a5fa916cf045bf19: Generated new TSK 0
I20260620 23:09:09.889245 10421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:09:09.891952 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41621
--builtin_ntp_servers=127.1.32.84:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 23:09:09.970016 10431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:09.970232 10431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:09.970265 10431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:09.971400 10431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:09.971452 10431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:09.971488 10431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:09:09.972743 10431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:09.972985 10431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:09.973284 10431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:09.978859 10436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:09.978920 10437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:09.979034 10439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:09.979133 10431 server_base.cc:1034] running on GCE node
I20260620 23:09:09.979390 10431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:09.979606 10431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:09.980789 10431 hybrid_clock.cc:648] HybridClock initialized: now 1781996949980766 us; error 41 us; skew 500 ppm
I20260620 23:09:09.981938 10431 webserver.cc:460] Webserver started at http://127.1.32.65:35489/ using document root <none> and password file <none>
I20260620 23:09:09.982167 10431 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:09.982230 10431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:09.982379 10431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:09.983403 10431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "f9f9666b0f654177a984bdf49017a132"
format_stamp: "Formatted at 2026-06-20 23:09:09 on dist-test-slave-fzjm"
server_key: "b5ef18220fb7dde93afb0a280cb0302c"
server_key_iv: "f5437952e35222c1791f5b2dc09dbd28"
server_key_version: "encryptionkey@0"
I20260620 23:09:09.983716 10431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "f9f9666b0f654177a984bdf49017a132"
format_stamp: "Formatted at 2026-06-20 23:09:09 on dist-test-slave-fzjm"
server_key: "b5ef18220fb7dde93afb0a280cb0302c"
server_key_iv: "f5437952e35222c1791f5b2dc09dbd28"
server_key_version: "encryptionkey@0"
I20260620 23:09:09.985077 10431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:09:09.985796 10445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:09.985975 10431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:09:09.986083 10431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "f9f9666b0f654177a984bdf49017a132"
format_stamp: "Formatted at 2026-06-20 23:09:09 on dist-test-slave-fzjm"
server_key: "b5ef18220fb7dde93afb0a280cb0302c"
server_key_iv: "f5437952e35222c1791f5b2dc09dbd28"
server_key_version: "encryptionkey@0"
I20260620 23:09:09.986157 10431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:10.002997 10431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:10.003288 10431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:10.003430 10431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:10.003659 10431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:10.004000 10431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:10.004055 10431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.004097 10431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:10.004127 10431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.009773 10431 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:35147
I20260620 23:09:10.009851 10558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:35147 every 8 connection(s)
I20260620 23:09:10.010185 10431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:09:10.015295 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.010933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:32857 (local address 127.1.32.126:41621)
0620 23:09:10.010984 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.010987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.011279 (+ 292us) server_negotiation.cc:407] Connection header received
0620 23:09:10.011744 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.011745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.011759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.011813 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.012436 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.012958 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.013940 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.014115 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.014182 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:10.015058 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:10.015059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:10.015059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:10.015063 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:10.015074 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:10.015077 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:10.015117 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:10.015219 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:09:10.015238 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:09:10.015538 10561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.010943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41621 (local address 127.1.32.65:32857)
0620 23:09:10.011149 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:10.011187 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:10.011663 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:10.011865 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:10.011868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:10.011876 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:10.012363 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:10.012367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.013033 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.013034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:10.013861 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:10.013863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.014208 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.014209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:10.014321 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:10.014936 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:09:10.014937 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:10.014968 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:09:10.014990 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:10.015144 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:10.015147 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:10.015149 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:09:10.015198 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:09:10.015286 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260620 23:09:10.016165 10559 heartbeater.cc:347] Connected to a master server at 127.1.32.126:41621
I20260620 23:09:10.016263 10559 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:10.016448 10559 heartbeater.cc:511] Master 127.1.32.126:41621 requested a full tablet report, sending...
I20260620 23:09:10.016446 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10431
I20260620 23:09:10.016531 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:09:10.016742 1153 external_mini_cluster.cc:1428] Setting key 9fc53208259df7c310d12002269a1a06
I20260620 23:09:10.017026 10371 ts_manager.cc:194] Registered new tserver with Master: f9f9666b0f654177a984bdf49017a132 (127.1.32.65:35147)
I20260620 23:09:10.017830 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41621
--builtin_ntp_servers=127.1.32.84:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 23:09:10.017820 10371 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.65:32857
W20260620 23:09:10.097937 10562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:10.098196 10562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:10.098234 10562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:10.099466 10562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:10.099532 10562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:10.099573 10562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:09:10.101481 10562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:10.101733 10562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:10.102092 10562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:10.107115 10568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.107139 10570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.107183 10567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:10.107389 10562 server_base.cc:1034] running on GCE node
I20260620 23:09:10.107589 10562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:10.107861 10562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:10.109014 10562 hybrid_clock.cc:648] HybridClock initialized: now 1781996950109002 us; error 23 us; skew 500 ppm
I20260620 23:09:10.110255 10562 webserver.cc:460] Webserver started at http://127.1.32.66:35923/ using document root <none> and password file <none>
I20260620 23:09:10.110481 10562 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:10.110548 10562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:10.110682 10562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:10.111735 10562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "a4d69a43184c473caec1f42ed58b695a"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "74854f891c5a4fab18ed3ad405ab6fb7"
server_key_iv: "4c9678e6c74438585e36eb1e7b4dcb9f"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.112120 10562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "a4d69a43184c473caec1f42ed58b695a"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "74854f891c5a4fab18ed3ad405ab6fb7"
server_key_iv: "4c9678e6c74438585e36eb1e7b4dcb9f"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.113354 10562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:09:10.114058 10576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.114215 10562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:09:10.114301 10562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "a4d69a43184c473caec1f42ed58b695a"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "74854f891c5a4fab18ed3ad405ab6fb7"
server_key_iv: "4c9678e6c74438585e36eb1e7b4dcb9f"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.114373 10562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:10.132550 10562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:10.132820 10562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:10.132957 10562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:10.133150 10562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:10.133476 10562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:10.133525 10562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.133556 10562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:10.133584 10562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.139269 10562 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:33413
I20260620 23:09:10.139307 10689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:33413 every 8 connection(s)
I20260620 23:09:10.139628 10562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:09:10.142316 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10562
I20260620 23:09:10.142390 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:09:10.142576 1153 external_mini_cluster.cc:1428] Setting key 5eaf65a33670658132c710fe2f81459d
I20260620 23:09:10.143518 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41621
--builtin_ntp_servers=127.1.32.84:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 23:09:10.145071 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.140313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:48273 (local address 127.1.32.126:41621)
0620 23:09:10.140395 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.140397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.140672 (+ 275us) server_negotiation.cc:407] Connection header received
0620 23:09:10.141195 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.141196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.141209 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.141257 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.141904 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.142497 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.143710 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.143909 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.143940 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:10.144800 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:10.144802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:10.144802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:10.144806 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:10.144822 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:10.144826 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:10.144877 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:10.144967 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 23:09:10.144993 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:09:10.145287 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.140323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41621 (local address 127.1.32.66:48273)
0620 23:09:10.140545 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:10.140576 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:10.141070 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:10.141335 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:10.141338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:10.141346 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:10.141811 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:10.141815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.142595 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.142596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:10.143590 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:10.143593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.144007 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.144007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:10.144108 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:10.144649 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:09:10.144649 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:10.144677 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:09:10.144697 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:10.144880 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:10.144884 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:10.144887 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:09:10.144970 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 23:09:10.145065 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260620 23:09:10.145682 10690 heartbeater.cc:347] Connected to a master server at 127.1.32.126:41621
I20260620 23:09:10.145773 10690 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:10.145962 10690 heartbeater.cc:511] Master 127.1.32.126:41621 requested a full tablet report, sending...
I20260620 23:09:10.146298 10371 ts_manager.cc:194] Registered new tserver with Master: a4d69a43184c473caec1f42ed58b695a (127.1.32.66:33413)
I20260620 23:09:10.146802 10371 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.66:48273
W20260620 23:09:10.223520 10693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:10.223713 10693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:10.223734 10693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:10.224833 10693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:10.224867 10693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:10.224891 10693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:09:10.226153 10693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:41621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:10.226328 10693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:10.226554 10693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:10.231258 10701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.231305 10699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.231272 10698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:10.231554 10693 server_base.cc:1034] running on GCE node
I20260620 23:09:10.231725 10693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:10.232018 10693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:10.233211 10693 hybrid_clock.cc:648] HybridClock initialized: now 1781996950233197 us; error 35 us; skew 500 ppm
I20260620 23:09:10.234300 10693 webserver.cc:460] Webserver started at http://127.1.32.67:33559/ using document root <none> and password file <none>
I20260620 23:09:10.234529 10693 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:10.234572 10693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:10.234709 10693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:10.235711 10693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "5ab1ba28f8ed495a8c1530e171cd1e89"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "70411e767501d4cdd898cb38238e6e56"
server_key_iv: "60a4843157f944ba9368fd8a6607f827"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.236052 10693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "5ab1ba28f8ed495a8c1530e171cd1e89"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "70411e767501d4cdd898cb38238e6e56"
server_key_iv: "60a4843157f944ba9368fd8a6607f827"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.237288 10693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:09:10.238020 10707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.238219 10693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:09:10.238302 10693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "5ab1ba28f8ed495a8c1530e171cd1e89"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "70411e767501d4cdd898cb38238e6e56"
server_key_iv: "60a4843157f944ba9368fd8a6607f827"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.238380 10693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:10.253332 10693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:10.253577 10693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:10.253700 10693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:10.253901 10693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:10.254189 10693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:10.254220 10693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.254267 10693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:10.254294 10693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.260063 10693 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:35683
I20260620 23:09:10.260110 10820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:35683 every 8 connection(s)
I20260620 23:09:10.260392 10693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:09:10.265260 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.261075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:49107 (local address 127.1.32.126:41621)
0620 23:09:10.261140 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.261143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.261378 (+ 235us) server_negotiation.cc:407] Connection header received
0620 23:09:10.261827 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.261828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.261836 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.261891 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.262496 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.263088 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.264035 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.264210 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.264267 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:10.265064 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:10.265065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:10.265065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:10.265068 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:10.265078 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:10.265081 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:10.265116 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:10.265181 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 23:09:10.265197 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:09:10.265481 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.261066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:41621 (local address 127.1.32.67:49107)
0620 23:09:10.261248 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:10.261287 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:10.261748 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:10.261940 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:10.261944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:10.261951 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:10.262409 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:10.262412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.263167 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.263168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:10.263961 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:10.263963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:10.264281 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:10.264282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:10.264384 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:10.264946 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:09:10.264947 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:10.264974 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:09:10.264994 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:10.265118 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:10.265120 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:10.265122 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:09:10.265170 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:09:10.265242 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260620 23:09:10.265975 10821 heartbeater.cc:347] Connected to a master server at 127.1.32.126:41621
I20260620 23:09:10.266064 10821 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:10.266234 10821 heartbeater.cc:511] Master 127.1.32.126:41621 requested a full tablet report, sending...
I20260620 23:09:10.266608 10371 ts_manager.cc:194] Registered new tserver with Master: 5ab1ba28f8ed495a8c1530e171cd1e89 (127.1.32.67:35683)
I20260620 23:09:10.267026 10371 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.67:49107
I20260620 23:09:10.268386 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10693
I20260620 23:09:10.268453 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:09:10.268635 1153 external_mini_cluster.cc:1428] Setting key 5a6b345c5f2bfee7f2b2e11209a4447c
I20260620 23:09:10.269444 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:09:10.271234 10411 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:10.270733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.1.32.126:41621)
0620 23:09:10.270771 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.270773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.270911 (+ 138us) server_negotiation.cc:407] Connection header received
0620 23:09:10.270991 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.270992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.271000 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.271048 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.271176 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49274: BlockingRecv error: recv got EOF from 127.0.0.1:49274 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:09:10.271281 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10431
I20260620 23:09:10.276113 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10562
I20260620 23:09:10.280784 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10693
I20260620 23:09:10.285261 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10340
2026-06-20T23:09:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (555 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T23:09:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:09:10Z Disabled control of system clock
I20260620 23:09:10.310703 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:44311
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:44311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:09:10.388703 10832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:10.388892 10832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:10.388911 10832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:10.389922 10832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:09:10.389953 10832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:10.389984 10832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:10.389997 10832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:09:10.390009 10832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:09:10.391335 10832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:44311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:44311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:10.391515 10832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:10.391733 10832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:10.396358 10837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.396373 10840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.396428 10838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:10.396415 10832 server_base.cc:1034] running on GCE node
I20260620 23:09:10.396806 10832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:10.397071 10832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:10.398206 10832 hybrid_clock.cc:648] HybridClock initialized: now 1781996950398202 us; error 38 us; skew 500 ppm
I20260620 23:09:10.399416 10832 webserver.cc:460] Webserver started at http://127.1.32.126:41495/ using document root <none> and password file <none>
I20260620 23:09:10.399644 10832 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:10.399709 10832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:10.399857 10832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:10.400915 10832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "f415735bda2f4e5a802f2c68c465d849"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "ac5e0ede0b2dfc7c37ba0b14fdc4f19b"
server_key_iv: "8e6eee3b830ea4175b0d17e627eef712"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.401455 10832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "f415735bda2f4e5a802f2c68c465d849"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "ac5e0ede0b2dfc7c37ba0b14fdc4f19b"
server_key_iv: "8e6eee3b830ea4175b0d17e627eef712"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.402839 10832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260620 23:09:10.403553 10846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.403725 10832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:09:10.403817 10832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "f415735bda2f4e5a802f2c68c465d849"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "ac5e0ede0b2dfc7c37ba0b14fdc4f19b"
server_key_iv: "8e6eee3b830ea4175b0d17e627eef712"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.403923 10832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:10.407217 10832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:10.407486 10832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:10.407625 10832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:10.411273 10832 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:44311
I20260620 23:09:10.411298 10898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:44311 every 8 connection(s)
I20260620 23:09:10.411604 10832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:09:10.412180 10899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:10.414435 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10832
I20260620 23:09:10.414534 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:09:10.414701 1153 external_mini_cluster.cc:1428] Setting key 867424f42107d6561d90213ed7eedbb1
I20260620 23:09:10.414703 10899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849: Bootstrap starting.
I20260620 23:09:10.415426 10899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:10.415714 10899 log.cc:824] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:10.416455 10899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849: No bootstrap required, opened a new log
I20260620 23:09:10.417285 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.415224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.1.32.126:44311)
0620 23:09:10.415432 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.415439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.415462 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:09:10.415964 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.415973 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.416022 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.416167 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.416840 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:10.416843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:10.416843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:10.416846 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:10.416883 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:10.416893 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:10.416959 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:10.417024 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 23:09:10.417091 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260620 23:09:10.418222 10899 raft_consensus.cc:348] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415735bda2f4e5a802f2c68c465d849" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44311 } }
I20260620 23:09:10.418366 10899 raft_consensus.cc:374] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:10.418411 10899 raft_consensus.cc:729] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f415735bda2f4e5a802f2c68c465d849, State: Initialized, Role: FOLLOWER
I20260620 23:09:10.418526 10899 consensus_queue.cc:260] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415735bda2f4e5a802f2c68c465d849" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44311 } }
I20260620 23:09:10.418615 10899 raft_consensus.cc:388] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:09:10.418658 10899 raft_consensus.cc:482] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:09:10.418706 10899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:10.419559 10899 raft_consensus.cc:504] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415735bda2f4e5a802f2c68c465d849" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44311 } }
I20260620 23:09:10.419675 10899 leader_election.cc:302] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f415735bda2f4e5a802f2c68c465d849; no voters:
I20260620 23:09:10.419818 10899 leader_election.cc:288] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:09:10.419847 10904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:09:10.419981 10904 raft_consensus.cc:686] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [term 1 LEADER]: Becoming Leader. State: Replica: f415735bda2f4e5a802f2c68c465d849, State: Running, Role: LEADER
I20260620 23:09:10.420069 10899 sys_catalog.cc:564] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:09:10.420132 10904 consensus_queue.cc:237] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415735bda2f4e5a802f2c68c465d849" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44311 } }
I20260620 23:09:10.420487 10905 sys_catalog.cc:455] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f415735bda2f4e5a802f2c68c465d849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415735bda2f4e5a802f2c68c465d849" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44311 } } }
I20260620 23:09:10.420487 10906 sys_catalog.cc:455] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f415735bda2f4e5a802f2c68c465d849. Latest consensus state: current_term: 1 leader_uuid: "f415735bda2f4e5a802f2c68c465d849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415735bda2f4e5a802f2c68c465d849" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 44311 } } }
I20260620 23:09:10.420698 10905 sys_catalog.cc:458] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [sys.catalog]: This master's current role is: LEADER
I20260620 23:09:10.420702 10906 sys_catalog.cc:458] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849 [sys.catalog]: This master's current role is: LEADER
I20260620 23:09:10.420907 10913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:09:10.421370 10913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:09:10.422734 10913 catalog_manager.cc:1347] Generated new cluster ID: 7d7f743eace44467bb954c4c779c25ad
I20260620 23:09:10.422791 10913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:09:10.435850 10913 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:09:10.436005 10913 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:09:10.441874 10913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f415735bda2f4e5a802f2c68c465d849: Generated new TSK 0
I20260620 23:09:10.442046 10913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:09:10.444645 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44311
--builtin_ntp_servers=127.1.32.84:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:09:10.522485 10923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:10.522734 10923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:10.522770 10923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:10.523977 10923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:10.524037 10923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:10.524075 10923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:09:10.525332 10923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:10.525571 10923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:10.525840 10923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:10.531427 10928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:10.531603 10923 server_base.cc:1034] running on GCE node
W20260620 23:09:10.531421 10929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.531433 10931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:10.531890 10923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:10.532157 10923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:10.533306 10923 hybrid_clock.cc:648] HybridClock initialized: now 1781996950533291 us; error 33 us; skew 500 ppm
I20260620 23:09:10.534494 10923 webserver.cc:460] Webserver started at http://127.1.32.65:41239/ using document root <none> and password file <none>
I20260620 23:09:10.534740 10923 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:10.534807 10923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:10.534972 10923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:10.536011 10923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "34de53a4a33a4437ae685b053c6ae73f"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "ccba5851a825e368a538248894133282"
server_key_iv: "517c86853662fafbabe71503bf1bb346"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.536369 10923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "34de53a4a33a4437ae685b053c6ae73f"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "ccba5851a825e368a538248894133282"
server_key_iv: "517c86853662fafbabe71503bf1bb346"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.537760 10923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:09:10.538529 10937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.538745 10923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:09:10.538857 10923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "34de53a4a33a4437ae685b053c6ae73f"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "ccba5851a825e368a538248894133282"
server_key_iv: "517c86853662fafbabe71503bf1bb346"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.539027 10923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:10.542181 10923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:10.542420 10923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:10.542550 10923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:10.542753 10923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:10.543071 10923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:10.543120 10923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.543149 10923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:10.543175 10923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.548754 10923 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:44691
I20260620 23:09:10.548854 11050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:44691 every 8 connection(s)
I20260620 23:09:10.549126 10923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:09:10.549358 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 10923
I20260620 23:09:10.549455 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:09:10.549642 1153 external_mini_cluster.cc:1428] Setting key e690727b820fc9428f120ea2be3918a8
I20260620 23:09:10.550649 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44311
--builtin_ntp_servers=127.1.32.84:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 23:09:10.552593 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.549811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:60537 (local address 127.1.32.126:44311)
0620 23:09:10.549859 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.549861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.550166 (+ 305us) server_negotiation.cc:407] Connection header received
0620 23:09:10.550837 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.550838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.550843 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.550871 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.552339 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:10.552341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:10.552341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:10.552346 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:10.552359 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:10.552364 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:10.552422 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:10.552512 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 23:09:10.552526 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:09:10.552824 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.549794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:44311 (local address 127.1.32.65:60537)
0620 23:09:10.550034 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:10.550074 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:10.550719 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:10.551081 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:10.551085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:10.551094 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:10.552153 (+ 1059us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:09:10.552153 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:10.552204 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:09:10.552230 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:10.552420 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:10.552424 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:10.552427 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:09:10.552521 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 23:09:10.552579 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260620 23:09:10.553263 11051 heartbeater.cc:347] Connected to a master server at 127.1.32.126:44311
I20260620 23:09:10.553359 11051 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:10.553504 11051 heartbeater.cc:511] Master 127.1.32.126:44311 requested a full tablet report, sending...
I20260620 23:09:10.553957 10863 ts_manager.cc:194] Registered new tserver with Master: 34de53a4a33a4437ae685b053c6ae73f (127.1.32.65:44691)
W20260620 23:09:10.632925 11054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:10.633095 11054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:10.633113 11054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:10.634132 11054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:10.634162 11054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:10.634186 11054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:09:10.635473 11054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:10.635645 11054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:10.635857 11054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:10.640547 11059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.640560 11062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.640559 11060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:10.640846 11054 server_base.cc:1034] running on GCE node
I20260620 23:09:10.641031 11054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:10.641260 11054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:10.642416 11054 hybrid_clock.cc:648] HybridClock initialized: now 1781996950642396 us; error 35 us; skew 500 ppm
I20260620 23:09:10.643709 11054 webserver.cc:460] Webserver started at http://127.1.32.66:38819/ using document root <none> and password file <none>
I20260620 23:09:10.643954 11054 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:10.643998 11054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:10.644135 11054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:10.645119 11054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "f946d45448e34d20b720904fa164f51c"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "1570d9ea6e1fa83154ba3c97c926cc78"
server_key_iv: "3ccb6bdd8c9d3b69d74aefa3710b7752"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.645442 11054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "f946d45448e34d20b720904fa164f51c"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "1570d9ea6e1fa83154ba3c97c926cc78"
server_key_iv: "3ccb6bdd8c9d3b69d74aefa3710b7752"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.646584 11054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:09:10.647337 11068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.647511 11054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.647578 11054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "f946d45448e34d20b720904fa164f51c"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "1570d9ea6e1fa83154ba3c97c926cc78"
server_key_iv: "3ccb6bdd8c9d3b69d74aefa3710b7752"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.647697 11054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:10.650748 11054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:10.651048 11054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:10.651180 11054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:10.651387 11054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:10.651714 11054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:10.651746 11054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.651793 11054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:10.651821 11054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.657401 11054 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:43385
I20260620 23:09:10.657449 11181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:43385 every 8 connection(s)
I20260620 23:09:10.657753 11054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:09:10.660672 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.658411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:35449 (local address 127.1.32.126:44311)
0620 23:09:10.658492 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.658494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.658682 (+ 188us) server_negotiation.cc:407] Connection header received
0620 23:09:10.659216 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.659218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.659224 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.659280 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.660428 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:10.660429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:10.660430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:10.660434 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:10.660447 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:10.660451 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:10.660506 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:10.660591 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 23:09:10.660604 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:09:10.660866 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.658379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:44311 (local address 127.1.32.66:35449)
0620 23:09:10.658564 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:10.658595 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:10.659097 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:10.659351 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:10.659354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:10.659361 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:10.660289 (+ 928us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:09:10.660289 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:10.660314 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:09:10.660339 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:10.660506 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:10.660511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:10.660514 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:09:10.660600 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:09:10.660662 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260620 23:09:10.661339 11182 heartbeater.cc:347] Connected to a master server at 127.1.32.126:44311
I20260620 23:09:10.661407 11182 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:10.661557 11182 heartbeater.cc:511] Master 127.1.32.126:44311 requested a full tablet report, sending...
I20260620 23:09:10.661919 10863 ts_manager.cc:194] Registered new tserver with Master: f946d45448e34d20b720904fa164f51c (127.1.32.66:43385)
I20260620 23:09:10.665427 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 11054
I20260620 23:09:10.665518 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:09:10.665701 1153 external_mini_cluster.cc:1428] Setting key 3f5af3c04435821b7e9016bde30ce652
I20260620 23:09:10.666512 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44311
--builtin_ntp_servers=127.1.32.84:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:09:10.744768 11185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:10.744958 11185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:10.744979 11185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:10.745999 11185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:10.746031 11185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:10.746055 11185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:09:10.747318 11185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:44311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:10.747498 11185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:10.747711 11185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:10.752411 11193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.752516 11191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:10.752417 11190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:10.752691 11185 server_base.cc:1034] running on GCE node
I20260620 23:09:10.752942 11185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:10.753177 11185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:10.754334 11185 hybrid_clock.cc:648] HybridClock initialized: now 1781996950754323 us; error 26 us; skew 500 ppm
I20260620 23:09:10.755527 11185 webserver.cc:460] Webserver started at http://127.1.32.67:42791/ using document root <none> and password file <none>
I20260620 23:09:10.755759 11185 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:10.755823 11185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:10.755965 11185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:10.756933 11185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "614dfacb4abe409cad6bf6365c3e4eb5"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "3ade6e98f54364ae4f198d15914962f2"
server_key_iv: "fe864d5f990fc3aa5d781f71ab622daf"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.757275 11185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "614dfacb4abe409cad6bf6365c3e4eb5"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "3ade6e98f54364ae4f198d15914962f2"
server_key_iv: "fe864d5f990fc3aa5d781f71ab622daf"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.758605 11185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:09:10.759366 11199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.759531 11185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.759618 11185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "614dfacb4abe409cad6bf6365c3e4eb5"
format_stamp: "Formatted at 2026-06-20 23:09:10 on dist-test-slave-fzjm"
server_key: "3ade6e98f54364ae4f198d15914962f2"
server_key_iv: "fe864d5f990fc3aa5d781f71ab622daf"
server_key_version: "encryptionkey@0"
I20260620 23:09:10.759696 11185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:10.762640 11185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:10.762924 11185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:10.763051 11185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:10.763243 11185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:10.763520 11185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:10.763568 11185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.763605 11185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:10.763631 11185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:10.769032 11185 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:41199
I20260620 23:09:10.769064 11312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:41199 every 8 connection(s)
I20260620 23:09:10.769388 11185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:09:10.770835 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 11185
I20260620 23:09:10.770934 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:09:10.771131 1153 external_mini_cluster.cc:1428] Setting key 10f444b2df694e846533a73fbb6348d8
I20260620 23:09:10.772178 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.770055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:33995 (local address 127.1.32.126:44311)
0620 23:09:10.770105 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.770108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.770361 (+ 253us) server_negotiation.cc:407] Connection header received
0620 23:09:10.770752 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.770754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.770761 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.770839 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.772011 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:10.772011 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:10.772012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:09:10.772015 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:10.772029 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:09:10.772033 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:10.772074 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:10.772120 (+ 46us) server_negotiation.cc:299] Negotiation successful
0620 23:09:10.772131 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:09:10.772339 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:10.770052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:44311 (local address 127.1.32.67:33995)
0620 23:09:10.770255 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:10.770283 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:10.770666 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:10.770933 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:10.770938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:10.770948 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:10.771889 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:09:10.771889 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:10.771914 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:09:10.771938 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:10.772073 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:10.772075 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:10.772077 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:09:10.772120 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 23:09:10.772162 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260620 23:09:10.772776 11313 heartbeater.cc:347] Connected to a master server at 127.1.32.126:44311
I20260620 23:09:10.772868 11313 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:10.773000 11313 heartbeater.cc:511] Master 127.1.32.126:44311 requested a full tablet report, sending...
I20260620 23:09:10.773320 10863 ts_manager.cc:194] Registered new tserver with Master: 614dfacb4abe409cad6bf6365c3e4eb5 (127.1.32.67:41199)
I20260620 23:09:10.782590 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:09:10.784407 10902 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:09:10.783991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.1.32.126:44311)
0620 23:09:10.784057 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:10.784059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:10.784092 (+ 33us) server_negotiation.cc:407] Connection header received
0620 23:09:10.784158 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:10.784158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:10.784162 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:10.784209 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:10.784343 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57646: BlockingRecv error: recv got EOF from 127.0.0.1:57646 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:09:10.784483 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10923
I20260620 23:09:10.789182 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 11054
I20260620 23:09:10.793295 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 11185
I20260620 23:09:10.796939 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 10832
2026-06-20T23:09:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (511 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:09:10 dist-test-slave-fzjm krb5kdc[11323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:09:10 dist-test-slave-fzjm krb5kdc[11323](info): set up 2 sockets
Jun 20 23:09:10 dist-test-slave-fzjm krb5kdc[11323](info): commencing operation
krb5kdc: starting...
W20260620 23:09:11.811532 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:09:11 dist-test-slave-fzjm krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:09:11.919391 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:42051
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:42051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:09:11.996290 11336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:11.996484 11336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:11.996503 11336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:11.997489 11336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:09:11.997521 11336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:11.997534 11336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:11.997547 11336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:09:11.997558 11336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:09:11.998958 11336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:42051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:42051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:11.999166 11336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:11.999323 11336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:12.004202 11342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:12.004312 11341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:12.004202 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:12.004407 11336 server_base.cc:1034] running on GCE node
I20260620 23:09:12.004632 11336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:12.004921 11336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:12.006093 11336 hybrid_clock.cc:648] HybridClock initialized: now 1781996952006065 us; error 29 us; skew 500 ppm
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:12.008481 11336 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:09:12.008894 11336 webserver.cc:460] Webserver started at http://127.1.32.126:33889/ using document root <none> and password file <none>
I20260620 23:09:12.009078 11336 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:12.009115 11336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:12.009207 11336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:12.010128 11336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "3ed89eda8ae5443288690714c4ee43e4"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "463168ecdaae8afa3f15fb50c4d91bf5"
server_key_iv: "a9cdcf3e48deaed7fcdda3195d1bbe50"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.010427 11336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "3ed89eda8ae5443288690714c4ee43e4"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "463168ecdaae8afa3f15fb50c4d91bf5"
server_key_iv: "a9cdcf3e48deaed7fcdda3195d1bbe50"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.011754 11336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:09:12.012435 11351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.012642 11336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:09:12.012728 11336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "3ed89eda8ae5443288690714c4ee43e4"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "463168ecdaae8afa3f15fb50c4d91bf5"
server_key_iv: "a9cdcf3e48deaed7fcdda3195d1bbe50"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.012781 11336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:12.032212 11336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:12.034624 11336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:12.034737 11336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:12.038659 11336 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:42051
I20260620 23:09:12.038733 11403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:42051 every 8 connection(s)
I20260620 23:09:12.039089 11336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:09:12.040066 11404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:12.042277 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4: Bootstrap starting.
I20260620 23:09:12.042851 11404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:12.043171 11404 log.cc:824] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:12.043571 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 11336
I20260620 23:09:12.043702 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:09:12.043907 1153 external_mini_cluster.cc:1428] Setting key 6c1b42c6f084a0d0153fd17aeef331df
I20260620 23:09:12.044044 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4: No bootstrap required, opened a new log
I20260620 23:09:12.046191 11404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed89eda8ae5443288690714c4ee43e4" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42051 } }
I20260620 23:09:12.046360 11404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:12.046398 11404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed89eda8ae5443288690714c4ee43e4, State: Initialized, Role: FOLLOWER
I20260620 23:09:12.046550 11404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed89eda8ae5443288690714c4ee43e4" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42051 } }
I20260620 23:09:12.046622 11404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:09:12.046660 11404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:09:12.046702 11404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:12.047369 11404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed89eda8ae5443288690714c4ee43e4" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42051 } }
I20260620 23:09:12.047465 11404 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed89eda8ae5443288690714c4ee43e4; no voters:
I20260620 23:09:12.047586 11404 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:09:12.047642 11409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:09:12.047784 11409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed89eda8ae5443288690714c4ee43e4, State: Running, Role: LEADER
I20260620 23:09:12.047791 11404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:09:12.047921 11409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed89eda8ae5443288690714c4ee43e4" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42051 } }
I20260620 23:09:12.048280 11411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ed89eda8ae5443288690714c4ee43e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed89eda8ae5443288690714c4ee43e4" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42051 } } }
I20260620 23:09:12.048385 11411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [sys.catalog]: This master's current role is: LEADER
I20260620 23:09:12.049499 11410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed89eda8ae5443288690714c4ee43e4. Latest consensus state: current_term: 1 leader_uuid: "3ed89eda8ae5443288690714c4ee43e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed89eda8ae5443288690714c4ee43e4" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42051 } } }
I20260620 23:09:12.049638 11410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4 [sys.catalog]: This master's current role is: LEADER
W20260620 23:09:12.049686 11425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:09:12.049775 11425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:09:12.050297 11416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:09:12.050493 11416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:09:12.052148 11416 catalog_manager.cc:1347] Generated new cluster ID: e60ab6b6077f4498a019bdb74ba803c1
I20260620 23:09:12.052198 11416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:09:12.056231 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.044407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.1.32.126:42051)
0620 23:09:12.044603 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.044620 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.044654 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:09:12.045417 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.045425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.045489 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.045657 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:12.046203 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.047754 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.048867 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.049477 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.049901 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.052346 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:12.052350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:12.052351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:12.052362 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:12.054428 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.054822 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.054825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.054825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.054953 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.055157 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.055157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.055158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.055470 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:12.055635 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.055709 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.055856 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260620 23:09:12.058003 11416 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:09:12.058656 11416 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:09:12.061728 11416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4: Generated new TSK 0
I20260620 23:09:12.061877 11416 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:09:12.089815 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42051
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:09:12.168690 11430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:12.168938 11430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:12.168972 11430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:12.170032 11430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:12.170096 11430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:12.170137 11430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:09:12.171805 11430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:12.172211 11430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:12.172515 11430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:12.177207 11436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:12.177202 11435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:12.177493 11430 server_base.cc:1034] running on GCE node
W20260620 23:09:12.177255 11438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:12.177805 11430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:12.178093 11430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:12.179270 11430 hybrid_clock.cc:648] HybridClock initialized: now 1781996952179257 us; error 30 us; skew 500 ppm
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:12.181607 11430 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:09:12.181990 11430 webserver.cc:460] Webserver started at http://127.1.32.65:33563/ using document root <none> and password file <none>
I20260620 23:09:12.182209 11430 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:12.182269 11430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:12.182418 11430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:12.183508 11430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "e607425bbf58452ca4be18e604349a5d"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "635f20676f50e149b53f19526dd6a4b7"
server_key_iv: "7ed444f55b5954736b20b94218ce127f"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.183794 11430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "e607425bbf58452ca4be18e604349a5d"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "635f20676f50e149b53f19526dd6a4b7"
server_key_iv: "7ed444f55b5954736b20b94218ce127f"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.185328 11430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:09:12.185945 11445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.186107 11430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:09:12.186208 11430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "e607425bbf58452ca4be18e604349a5d"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "635f20676f50e149b53f19526dd6a4b7"
server_key_iv: "7ed444f55b5954736b20b94218ce127f"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.186281 11430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:12.193389 11430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:12.195062 11430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:12.195200 11430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:12.195422 11430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:12.195709 11430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:12.195761 11430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.195801 11430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:12.195825 11430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.201531 11430 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:44007
I20260620 23:09:12.201593 11558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:44007 every 8 connection(s)
I20260620 23:09:12.201954 11430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:09:12.204452 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 11430
I20260620 23:09:12.204523 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:09:12.204700 1153 external_mini_cluster.cc:1428] Setting key 49750a4d457acb639f15337847fc8e9d
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:09:12.212029 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.202927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:60187 (local address 127.1.32.126:42051)
0620 23:09:12.202967 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.202970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.203204 (+ 234us) server_negotiation.cc:407] Connection header received
0620 23:09:12.203728 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.203729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.203742 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.203798 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:12.204745 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.205413 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.206410 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.206628 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.206707 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.208868 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:12.208869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:12.208870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:12.208877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:12.210502 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.211001 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.211002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.211002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.211065 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.211327 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.211328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.211328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.211488 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:12.211560 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.211812 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.211919 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:09:12.212097 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.202874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42051 (local address 127.1.32.65:60187)
0620 23:09:12.203085 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.203123 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.203600 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.203875 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.203878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.204413 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:12.204652 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.204657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.205495 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.205496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.206309 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.206311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.206716 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.206717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.206831 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.207452 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:12.207468 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:12.208780 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:12.210617 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:12.210623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:12.210625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:12.210847 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:12.211136 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:12.211137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:12.211137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:12.211254 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:12.211554 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:12.211558 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:12.211671 (+ 113us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.211747 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.211876 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260620 23:09:12.212626 11559 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42051
I20260620 23:09:12.212721 11559 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:12.212910 11559 heartbeater.cc:511] Master 127.1.32.126:42051 requested a full tablet report, sending...
I20260620 23:09:12.213445 11368 ts_manager.cc:194] Registered new tserver with Master: e607425bbf58452ca4be18e604349a5d (127.1.32.65:44007)
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:09:12.214098 11368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:60187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:09:12.229449 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42051
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:09:12.310051 11564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:12.310299 11564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:12.310335 11564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:12.311496 11564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:12.311559 11564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:12.311600 11564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:09:12.313087 11564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:12.313357 11564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:12.313592 11564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:12.318699 11572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:12.318653 11570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:12.318629 11569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:12.318828 11564 server_base.cc:1034] running on GCE node
I20260620 23:09:12.319036 11564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:12.319271 11564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:12.320416 11564 hybrid_clock.cc:648] HybridClock initialized: now 1781996952320403 us; error 28 us; skew 500 ppm
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:12.322999 11564 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:09:12.323407 11564 webserver.cc:460] Webserver started at http://127.1.32.66:45013/ using document root <none> and password file <none>
I20260620 23:09:12.323637 11564 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:12.323701 11564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:12.323854 11564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:12.324987 11564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "f50c42ecccc442bfb698b19c72acf8bf"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "dee1401baa6b3ded68a8fb8c1ebb89d0"
server_key_iv: "6562fb29da814fba05bc875ca58724de"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.325379 11564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "f50c42ecccc442bfb698b19c72acf8bf"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "dee1401baa6b3ded68a8fb8c1ebb89d0"
server_key_iv: "6562fb29da814fba05bc875ca58724de"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.326713 11564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:09:12.327593 11579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.327808 11564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:09:12.327907 11564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "f50c42ecccc442bfb698b19c72acf8bf"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "dee1401baa6b3ded68a8fb8c1ebb89d0"
server_key_iv: "6562fb29da814fba05bc875ca58724de"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.327991 11564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:12.344154 11564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:12.345896 11564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:12.346050 11564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:12.346267 11564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:12.346549 11564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:12.346601 11564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.346640 11564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:12.346668 11564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.352737 11564 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:46045
I20260620 23:09:12.352778 11692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:46045 every 8 connection(s)
I20260620 23:09:12.353075 11564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:09:12.354250 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 11564
I20260620 23:09:12.354350 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:09:12.354532 1153 external_mini_cluster.cc:1428] Setting key f4cb6a31804117c74282d1a63491a3fa
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
I20260620 23:09:12.363268 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.353786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:46077 (local address 127.1.32.126:42051)
0620 23:09:12.353853 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.353854 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.354071 (+ 217us) server_negotiation.cc:407] Connection header received
0620 23:09:12.354782 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.354783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.354795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.354845 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:12.356121 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.356614 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.357620 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.357830 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.357901 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.360340 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:12.360342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:12.360342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:12.360351 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:12.361884 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.362252 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.362253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.362254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.362318 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.362537 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.362538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.362538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.362683 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:12.362754 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.363033 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.363119 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:09:12.363409 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.353758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42051 (local address 127.1.32.66:46077)
0620 23:09:12.353929 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.353968 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.354670 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.354964 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.354969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.355741 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:12.356027 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.356034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.356734 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.356736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.357520 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.357522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.357917 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.357918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.358039 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.358636 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:12.358644 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:12.360247 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:12.361961 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:12.361966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:12.361968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:12.362174 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:12.362384 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:12.362385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:12.362385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:12.362473 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:12.362747 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:12.362751 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:12.362861 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.362989 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.363152 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260620 23:09:12.363852 11693 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42051
I20260620 23:09:12.363961 11693 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:12.364137 11693 heartbeater.cc:511] Master 127.1.32.126:42051 requested a full tablet report, sending...
I20260620 23:09:12.364540 11368 ts_manager.cc:194] Registered new tserver with Master: f50c42ecccc442bfb698b19c72acf8bf (127.1.32.66:46045)
I20260620 23:09:12.364943 11368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:46077
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:09:12.379444 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42051
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:09:12.456126 11698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:12.456311 11698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:12.456333 11698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:12.457427 11698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:12.457468 11698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:12.457494 11698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:09:12.458853 11698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:12.459077 11698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:12.459296 11698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:12.464092 11704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:12.464072 11703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:12.464097 11706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:12.464241 11698 server_base.cc:1034] running on GCE node
I20260620 23:09:12.464413 11698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:12.464650 11698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:12.465837 11698 hybrid_clock.cc:648] HybridClock initialized: now 1781996952465815 us; error 31 us; skew 500 ppm
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:12.468465 11698 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:09:12.468860 11698 webserver.cc:460] Webserver started at http://127.1.32.67:37557/ using document root <none> and password file <none>
I20260620 23:09:12.469080 11698 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:12.469146 11698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:12.469291 11698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:09:12.470309 11698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "a86b71a699ed4c179b87d8eab82d4f45"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "9d48a6f23e4958507bb10042e4ca580f"
server_key_iv: "d67b5ef684e503bcd71818650aecb4ee"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.470666 11698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "a86b71a699ed4c179b87d8eab82d4f45"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "9d48a6f23e4958507bb10042e4ca580f"
server_key_iv: "d67b5ef684e503bcd71818650aecb4ee"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.472003 11698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:09:12.472692 11713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.472853 11698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:09:12.472952 11698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "a86b71a699ed4c179b87d8eab82d4f45"
format_stamp: "Formatted at 2026-06-20 23:09:12 on dist-test-slave-fzjm"
server_key: "9d48a6f23e4958507bb10042e4ca580f"
server_key_iv: "d67b5ef684e503bcd71818650aecb4ee"
server_key_version: "encryptionkey@0"
I20260620 23:09:12.473034 11698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:12.480741 11698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:12.482617 11698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:12.482771 11698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:12.483011 11698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:12.483290 11698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:12.483338 11698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.483374 11698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:12.483399 11698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:12.488808 11698 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:38965
I20260620 23:09:12.488862 11826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:38965 every 8 connection(s)
I20260620 23:09:12.489183 11698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:09:12.493361 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 11698
I20260620 23:09:12.493472 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:09:12.493734 1153 external_mini_cluster.cc:1428] Setting key b7628cd81463727a519b2a68cee07225
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:09:12.498286 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.489886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:44407 (local address 127.1.32.126:42051)
0620 23:09:12.489929 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.489931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.490137 (+ 206us) server_negotiation.cc:407] Connection header received
0620 23:09:12.490550 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.490551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.490560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.490614 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:12.491528 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.491967 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.492993 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.493176 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.493229 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.495720 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:12.495722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:12.495722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:12.495728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:12.497046 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.497487 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.497488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.497489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.497542 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.497778 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.497779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.497779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.497875 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:12.497925 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.498165 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.498226 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:09:12.498440 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.489836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42051 (local address 127.1.32.67:44407)
0620 23:09:12.489987 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.490022 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.490470 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.490684 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.490687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.491273 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:12.491444 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.491449 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.492068 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.492069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.492915 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.492917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.493259 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.493260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.493404 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.494005 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:12.494012 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:12.495633 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:12.497136 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:12.497141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:12.497143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:12.497400 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:12.497617 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:12.497618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:12.497618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:12.497716 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:12.497932 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:12.497936 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:12.498037 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.498141 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.498253 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260620 23:09:12.498951 11827 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42051
I20260620 23:09:12.499038 11827 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:12.499253 11827 heartbeater.cc:511] Master 127.1.32.126:42051 requested a full tablet report, sending...
I20260620 23:09:12.499682 11368 ts_manager.cc:194] Registered new tserver with Master: a86b71a699ed4c179b87d8eab82d4f45 (127.1.32.67:38965)
I20260620 23:09:12.500003 11368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:44407
I20260620 23:09:12.505601 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:09:12.512221 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.506835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.1.32.126:42051)
0620 23:09:12.506925 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.506927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.507066 (+ 139us) server_negotiation.cc:407] Connection header received
0620 23:09:12.507092 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.507092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.507101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.507148 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:12.507679 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.508134 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.508827 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.509003 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.509057 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.510040 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:12.510041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:12.510041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:12.510046 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:12.511332 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.511619 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.511620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.511620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.511662 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:12.511809 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:12.511809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:12.511810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:12.511922 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:12.511987 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.512067 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.512135 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:09:12.513456 11368 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:09:12.514930 11368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:09:12.520804 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.516834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:44007 (local address 127.0.0.1:33558)
0620 23:09:12.517142 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.517151 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.517203 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.517495 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.517496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.517497 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:12.517522 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.517523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.518792 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.518793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.519596 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.519599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.520530 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.520531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.520609 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.520622 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.520683 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.520732 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":2,"thread_start_us":42,"threads_started":1}
I20260620 23:09:12.520910 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.516831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:38965 (local address 127.0.0.1:40944)
0620 23:09:12.516990 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.517026 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.517085 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.517337 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.517340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.517342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:12.517398 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.517399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.518663 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.518664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.519823 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.519826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.520611 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.520612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.520684 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.520688 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.520743 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.520784 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":48,"threads_started":1}
I20260620 23:09:12.521137 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.516880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.1.32.67:38965)
0620 23:09:12.517139 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.517143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.517153 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:09:12.517185 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.517187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.517200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.517266 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:12.517470 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.518554 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.519901 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.520528 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.520594 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.520657 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.520753 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.520786 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":85,"threads_started":1}
I20260620 23:09:12.521153 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.516957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33558 (local address 127.1.32.65:44007)
0620 23:09:12.517350 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.517353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.517362 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:09:12.517392 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.517393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.517404 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.517505 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:12.517574 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.518582 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.519682 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.520377 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.520442 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.520495 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.521003 (+ 508us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.521041 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":73,"threads_started":1}
I20260620 23:09:12.521687 11761 tablet_service.cc:1467] Processing CreateTablet for tablet bfabacc3737544b0a5a25ac622f81047 (DEFAULT_TABLE table=test-table [id=a7b9f0a7321e4d48a1e9f7bcb0d5bd44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:09:12.521754 11493 tablet_service.cc:1467] Processing CreateTablet for tablet bfabacc3737544b0a5a25ac622f81047 (DEFAULT_TABLE table=test-table [id=a7b9f0a7321e4d48a1e9f7bcb0d5bd44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:09:12.522184 11493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfabacc3737544b0a5a25ac622f81047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:12.522219 11761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfabacc3737544b0a5a25ac622f81047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:12.522383 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.517033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:46045 (local address 127.0.0.1:59450)
0620 23:09:12.518695 (+ 1662us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.518704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.518751 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.519001 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.519002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.519004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:12.519040 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.519042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.520176 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.520177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.521055 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.521058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.522169 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.522170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.522232 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.522236 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.522286 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.522324 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":64,"threads_started":1}
I20260620 23:09:12.522615 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.517147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.1.32.66:46045)
0620 23:09:12.517300 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.517303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.518771 (+ 1468us) server_negotiation.cc:407] Connection header received
0620 23:09:12.518813 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.518815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.518828 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.518928 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:12.519142 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.520084 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.521137 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.522015 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.522083 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.522132 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.522476 (+ 344us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.522524 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260620 23:09:12.524776 11842 tablet_bootstrap.cc:492] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Bootstrap starting.
I20260620 23:09:12.524776 11841 tablet_bootstrap.cc:492] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d: Bootstrap starting.
I20260620 23:09:12.525317 11842 tablet_bootstrap.cc:654] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:12.525395 11841 tablet_bootstrap.cc:654] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:12.525537 11627 tablet_service.cc:1467] Processing CreateTablet for tablet bfabacc3737544b0a5a25ac622f81047 (DEFAULT_TABLE table=test-table [id=a7b9f0a7321e4d48a1e9f7bcb0d5bd44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:09:12.525835 11842 log.cc:824] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:12.525853 11627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfabacc3737544b0a5a25ac622f81047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:12.525915 11841 log.cc:824] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:12.526604 11841 tablet_bootstrap.cc:492] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d: No bootstrap required, opened a new log
I20260620 23:09:12.526587 11842 tablet_bootstrap.cc:492] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: No bootstrap required, opened a new log
I20260620 23:09:12.526683 11842 ts_tablet_manager.cc:1397] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:09:12.526685 11841 ts_tablet_manager.cc:1397] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:09:12.527994 11841 raft_consensus.cc:348] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.528110 11841 raft_consensus.cc:374] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:12.528337 11841 raft_consensus.cc:729] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e607425bbf58452ca4be18e604349a5d, State: Initialized, Role: FOLLOWER
I20260620 23:09:12.528502 11841 consensus_queue.cc:260] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.528550 11842 raft_consensus.cc:348] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.528703 11842 raft_consensus.cc:374] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:12.528741 11842 raft_consensus.cc:729] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86b71a699ed4c179b87d8eab82d4f45, State: Initialized, Role: FOLLOWER
I20260620 23:09:12.528791 11841 ts_tablet_manager.cc:1428] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:09:12.528851 11842 consensus_queue.cc:260] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.528898 11559 heartbeater.cc:503] Master 127.1.32.126:42051 was elected leader, sending a full tablet report...
I20260620 23:09:12.529100 11842 ts_tablet_manager.cc:1428] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:09:12.529119 11827 heartbeater.cc:503] Master 127.1.32.126:42051 was elected leader, sending a full tablet report...
I20260620 23:09:12.529129 11846 tablet_bootstrap.cc:492] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Bootstrap starting.
I20260620 23:09:12.529745 11846 tablet_bootstrap.cc:654] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:12.530304 11846 log.cc:824] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:12.531028 11846 tablet_bootstrap.cc:492] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: No bootstrap required, opened a new log
I20260620 23:09:12.531098 11846 ts_tablet_manager.cc:1397] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:09:12.532368 11846 raft_consensus.cc:348] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.532500 11846 raft_consensus.cc:374] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:12.532543 11846 raft_consensus.cc:729] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50c42ecccc442bfb698b19c72acf8bf, State: Initialized, Role: FOLLOWER
I20260620 23:09:12.532647 11846 consensus_queue.cc:260] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.532902 11846 ts_tablet_manager.cc:1428] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:09:12.532933 11693 heartbeater.cc:503] Master 127.1.32.126:42051 was elected leader, sending a full tablet report...
W20260620 23:09:12.603610 11694 tablet.cc:2369] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:09:12.622032 11849 raft_consensus.cc:482] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:09:12.622206 11849 raft_consensus.cc:504] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.622592 11849 leader_election.cc:288] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a86b71a699ed4c179b87d8eab82d4f45 (127.1.32.67:38965), e607425bbf58452ca4be18e604349a5d (127.1.32.65:44007)
I20260620 23:09:12.626494 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.622871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:48995 (local address 127.1.32.67:38965)
0620 23:09:12.622950 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.622952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.622959 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:09:12.622987 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.622988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.623000 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.623055 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:12.623243 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.624021 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.625558 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.626155 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.626226 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.626236 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.626396 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.626430 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 23:09:12.626494 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.622933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:35431 (local address 127.1.32.65:44007)
0620 23:09:12.622989 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.622991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.623015 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:09:12.623039 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.623040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.623050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.623100 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:12.623249 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.624022 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.625558 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.626155 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.626230 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.626238 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.626396 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.626431 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:09:12.626559 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.622751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:38965 (local address 127.1.32.66:48995)
0620 23:09:12.622811 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.622822 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.622876 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.623055 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.623056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.623057 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:12.623151 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.623153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.624123 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.624124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.625474 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.625475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.626243 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.626243 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.626318 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.626324 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.626378 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.626433 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":68}
I20260620 23:09:12.626569 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.622751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:44007 (local address 127.1.32.66:35431)
0620 23:09:12.622920 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.622930 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:12.622982 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:12.623121 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:12.623122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:12.623123 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:12.623179 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.623180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.624107 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.624108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.625466 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:12.625470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.626243 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.626243 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:12.626318 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.626324 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:09:12.626378 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:09:12.626433 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":125,"thread_start_us":83,"threads_started":1}
I20260620 23:09:12.626925 11513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfabacc3737544b0a5a25ac622f81047" candidate_uuid: "f50c42ecccc442bfb698b19c72acf8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e607425bbf58452ca4be18e604349a5d" is_pre_election: true
I20260620 23:09:12.626941 11781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfabacc3737544b0a5a25ac622f81047" candidate_uuid: "f50c42ecccc442bfb698b19c72acf8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a86b71a699ed4c179b87d8eab82d4f45" is_pre_election: true
I20260620 23:09:12.627090 11513 raft_consensus.cc:2454] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f50c42ecccc442bfb698b19c72acf8bf in term 0.
I20260620 23:09:12.627101 11781 raft_consensus.cc:2454] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f50c42ecccc442bfb698b19c72acf8bf in term 0.
I20260620 23:09:12.627295 11580 leader_election.cc:302] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e607425bbf58452ca4be18e604349a5d, f50c42ecccc442bfb698b19c72acf8bf; no voters:
I20260620 23:09:12.627430 11849 raft_consensus.cc:2781] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:09:12.627481 11849 raft_consensus.cc:482] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:09:12.627503 11849 raft_consensus.cc:3037] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:12.628140 11849 raft_consensus.cc:504] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.628280 11849 leader_election.cc:288] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [CANDIDATE]: Term 1 election: Requested vote from peers a86b71a699ed4c179b87d8eab82d4f45 (127.1.32.67:38965), e607425bbf58452ca4be18e604349a5d (127.1.32.65:44007)
I20260620 23:09:12.628402 11781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfabacc3737544b0a5a25ac622f81047" candidate_uuid: "f50c42ecccc442bfb698b19c72acf8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a86b71a699ed4c179b87d8eab82d4f45"
I20260620 23:09:12.628423 11513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfabacc3737544b0a5a25ac622f81047" candidate_uuid: "f50c42ecccc442bfb698b19c72acf8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e607425bbf58452ca4be18e604349a5d"
I20260620 23:09:12.628491 11513 raft_consensus.cc:3037] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:12.628491 11781 raft_consensus.cc:3037] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:12.629347 11781 raft_consensus.cc:2454] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f50c42ecccc442bfb698b19c72acf8bf in term 1.
I20260620 23:09:12.629347 11513 raft_consensus.cc:2454] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f50c42ecccc442bfb698b19c72acf8bf in term 1.
I20260620 23:09:12.629522 11583 leader_election.cc:302] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a86b71a699ed4c179b87d8eab82d4f45, f50c42ecccc442bfb698b19c72acf8bf; no voters:
I20260620 23:09:12.629626 11849 raft_consensus.cc:2781] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:09:12.629757 11849 raft_consensus.cc:686] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 1 LEADER]: Becoming Leader. State: Replica: f50c42ecccc442bfb698b19c72acf8bf, State: Running, Role: LEADER
I20260620 23:09:12.629846 11849 consensus_queue.cc:237] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } }
I20260620 23:09:12.630605 11366 catalog_manager.cc:5495] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf reported cstate change: term changed from 0 to 1, leader changed from <none> to f50c42ecccc442bfb698b19c72acf8bf (127.1.32.66). New cstate: current_term: 1 leader_uuid: "f50c42ecccc442bfb698b19c72acf8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f50c42ecccc442bfb698b19c72acf8bf" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46045 } health_report { overall_health: HEALTHY } } }
I20260620 23:09:12.635776 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.633515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.1.32.66:46045)
0620 23:09:12.633553 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:12.633554 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:12.633564 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:09:12.633606 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:12.633607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:12.633623 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:12.633682 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:09:12.633891 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.634329 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.635166 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:12.635337 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:12.635401 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:12.635515 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:09:12.635597 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:09:12.635662 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:12.635687 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 23:09:12.635704 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:09:12.637192 11781 raft_consensus.cc:1261] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 1 FOLLOWER]: Refusing update from remote peer f50c42ecccc442bfb698b19c72acf8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:09:12.637195 11513 raft_consensus.cc:1261] T bfabacc3737544b0a5a25ac622f81047 P e607425bbf58452ca4be18e604349a5d [term 1 FOLLOWER]: Refusing update from remote peer f50c42ecccc442bfb698b19c72acf8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:09:12.637497 11849 consensus_queue.cc:1035] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e607425bbf58452ca4be18e604349a5d" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 44007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:09:12.637614 11851 consensus_queue.cc:1035] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a86b71a699ed4c179b87d8eab82d4f45" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:09:12.640101 11855 mvcc.cc:205] Tried to move back new op lower bound from 7299059517999841280 to 7299059517972426752. Current Snapshot: MvccSnapshot[applied={T|T < 7299059517999841280}]
I20260620 23:09:12.641573 11366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:09:12.641647 11366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:09:12.642324 11366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ed89eda8ae5443288690714c4ee43e4: Sending DeleteTablet for 3 replicas of tablet bfabacc3737544b0a5a25ac622f81047
I20260620 23:09:12.642681 11627 tablet_service.cc:1514] Processing DeleteTablet for tablet bfabacc3737544b0a5a25ac622f81047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:09:12 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:59450
I20260620 23:09:12.642711 11493 tablet_service.cc:1514] Processing DeleteTablet for tablet bfabacc3737544b0a5a25ac622f81047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:09:12 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:33558
I20260620 23:09:12.642831 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 11430
I20260620 23:09:12.643517 11761 tablet_service.cc:1514] Processing DeleteTablet for tablet bfabacc3737544b0a5a25ac622f81047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:09:12 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:40944
I20260620 23:09:12.643832 11865 tablet_replica.cc:330] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: stopping tablet replica
I20260620 23:09:12.643895 11865 raft_consensus.cc:2229] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:09:12.643952 11865 raft_consensus.cc:2258] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:09:12.644088 11863 tablet_replica.cc:330] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: stopping tablet replica
I20260620 23:09:12.644176 11863 raft_consensus.cc:2229] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:09:12.644230 11863 raft_consensus.cc:2258] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:09:12.644559 11863 ts_tablet_manager.cc:1905] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:09:12.645272 11865 ts_tablet_manager.cc:1905] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:09:12.646049 11863 ts_tablet_manager.cc:1918] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:09:12.646130 11863 log.cc:1196] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/bfabacc3737544b0a5a25ac622f81047
I20260620 23:09:12.646406 11863 ts_tablet_manager.cc:1939] T bfabacc3737544b0a5a25ac622f81047 P f50c42ecccc442bfb698b19c72acf8bf: Deleting consensus metadata
I20260620 23:09:12.646773 11355 catalog_manager.cc:4841] TS f50c42ecccc442bfb698b19c72acf8bf (127.1.32.66:46045): tablet bfabacc3737544b0a5a25ac622f81047 (table test-table [id=a7b9f0a7321e4d48a1e9f7bcb0d5bd44]) successfully deleted
I20260620 23:09:12.647044 11865 ts_tablet_manager.cc:1918] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:09:12.647089 11865 log.cc:1196] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/bfabacc3737544b0a5a25ac622f81047
I20260620 23:09:12.647269 11865 ts_tablet_manager.cc:1939] T bfabacc3737544b0a5a25ac622f81047 P a86b71a699ed4c179b87d8eab82d4f45: Deleting consensus metadata
I20260620 23:09:12.647789 11355 catalog_manager.cc:4841] TS a86b71a699ed4c179b87d8eab82d4f45 (127.1.32.67:38965): tablet bfabacc3737544b0a5a25ac622f81047 (table test-table [id=a7b9f0a7321e4d48a1e9f7bcb0d5bd44]) successfully deleted
W20260620 23:09:12.649497 11352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:44007 (error 108)
I20260620 23:09:12.649626 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 11564
I20260620 23:09:12.651474 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:12.651346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:44007 (local address 127.0.0.1:33568)
0620 23:09:12.651402 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:12.651440 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:44007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 23:09:12.651584 11352 catalog_manager.cc:4584] TS e607425bbf58452ca4be18e604349a5d (127.1.32.65:44007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfabacc3737544b0a5a25ac622f81047: Network error: Client connection negotiation failed: client connection to 127.1.32.65:44007: connect: Connection refused (error 111)
I20260620 23:09:12.654500 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 11698
I20260620 23:09:12.659202 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 11336
2026-06-20T23:09:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1862 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11869](info): set up 2 sockets
Jun 20 23:09:12 dist-test-slave-fzjm krb5kdc[11869](info): commencing operation
krb5kdc: starting...
W20260620 23:09:13.667870 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:09:13 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:09:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/rangeradmin_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/rangeradmin_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/rangerlookup_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/rangerlookup_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
I20260620 23:09:13.817380 1153 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfTMrFo/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/postgres -l logfile start
2026-06-20 23:09:16.061 UTC [11899] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 23:09:16.061 UTC [11899] LOG: listening on IPv4 address "127.1.32.84", port 59817
2026-06-20 23:09:16.064 UTC [11899] LOG: listening on Unix socket "/tmp/.s.PGSQL.59817"
2026-06-20 23:09:16.099 UTC [11902] LOG: database system was shut down at 2026-06-20 23:09:14 UTC
2026-06-20 23:09:16.103 UTC [11899] LOG: database system is ready to accept connections
I20260620 23:09:17.011860 1153 mini_postgres.cc:96] Postgres bound to 59817
2026-06-20 23:09:17.017 UTC [11910] FATAL: database "slave" does not exist
127.1.32.84:59817 - accepting connections
I20260620 23:09:17.018337 1153 mini_ranger.cc:161] Starting Ranger...
I20260620 23:09:17.031756 1153 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 23:09:17.435523 1153 mini_ranger.cc:87] Created ranger Postgres database
I20260620 23:09:17.435616 1153 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-admin
2026-06-20 23:09:17,708 [I] DB FLAVOR :POSTGRES
2026-06-20 23:09:17,709 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:09:17,709 [I] Checking connection..
2026-06-20 23:09:17,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 23:09:17,958 [I] Checking connection passed.
2026-06-20 23:09:17,958 [I] --------- Verifying version history table ---------
2026-06-20 23:09:17,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:09:18,197 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:09:18,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:09:18,441 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:09:18,442 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 23:09:18,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 23:09:18.666 UTC [11974] WARNING: there is no transaction in progress
2026-06-20 23:09:18,671 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 23:09:18,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:09:18,889 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 23:09:18,889 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:09:18,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 23:09:19,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 23:09:19,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fzjm.c.gcp-upstream.internal','N') ;"
2026-06-20 23:09:19,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:09:19,827 [I] Table x_portal_user does not exist in database ranger
2026-06-20 23:09:19,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:09:20,075 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 23:09:20,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:09:20,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:09:20,548 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 23:09:20,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 23:09:21.465 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.479 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.494 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.508 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.523 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.539 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.589 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.596 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.609 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.618 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.627 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.640 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.649 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.655 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.660 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:21.666 UTC [12082] WARNING: there is no transaction in progress
2026-06-20 23:09:22,218 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 23:09:22,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 23:09:22,439 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 23:09:22,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:09:22,660 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 23:09:22,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:09:22,911 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 23:09:22,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:09:23,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:09:23,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fzjm.c.gcp-upstream.internal';"
2026-06-20 23:09:23,556 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 23:09:23,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 23:09:23,779 [I] DB_PATCHES have already been applied
I20260620 23:09:23.784718 1153 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfTMrFo/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 23:09:23.784801 1153 mini_ranger.cc:193] Using host: 127.1.32.84
I20260620 23:09:23.787707 1153 mini_ranger.cc:227] Ranger admin URL: http://127.1.32.84:37303
Jun 20, 2026 11:09:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:09:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:09:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:09:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:09:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:09:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:09:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 23:09:24.891945 1153 mini_ranger.cc:160] Time spent starting Ranger: real 7.874s user 0.001s sys 0.007s
Jun 20, 2026 11:09:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:09:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 20, 2026 11:09:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:09:44 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:09:44 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 23:09:45.443146 1153 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/rangerkms_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/rangerkms_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.32.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.32.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 23:09:45 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996985, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 23:09:45.523572 1153 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 23:09:45.532532 1153 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 23:09:46.269665 1153 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 23:09:46.269761 1153 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms
I20260620 23:09:46.269788 1153 mini_ranger_kms.cc:227] Using postgres at 127.1.32.84:59817
2026-06-20 23:09:46,435 [I] DB FLAVOR :POSTGRES
2026-06-20 23:09:46,436 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:09:46,436 [I] Checking connection
2026-06-20 23:09:46,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 23:09:46,677 [I] connection success
2026-06-20 23:09:46,677 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 23:09:46,677 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 23:09:46,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 23:09:46,893 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 23:09:46,893 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:09:46,893 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 23:09:46,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:59817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 23:09:47,143 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 23:09:47.578569 1153 mini_ranger_kms.cc:310] Created kms service
I20260620 23:09:47.783980 1153 mini_ranger_kms.cc:326] Created kudu user
I20260620 23:09:47.828459 1153 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 23:09:47 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996985, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:09:48.193658 1153 mini_ranger_kms.cc:384] Added ranger policy
I20260620 23:09:48.193771 1153 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 23:09:48.193792 1153 mini_ranger_kms.cc:240] Using host: 127.1.32.84
I20260620 23:09:48.197335 1153 mini_ranger_kms.cc:276] Ranger KMS PID: 12262
I20260620 23:09:48.197456 1153 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.32.84:36783
Jun 20, 2026 11:09:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:09:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:09:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:09:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:09:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:09:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 23:09:50.334314 1153 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.811s user 0.001s sys 0.148s
I20260620 23:09:50.334435 1153 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 23:09:50.334537 1153 mini_ranger_kms.cc:401] 127.1.32.84:36783/kms/v1/keys
Jun 20 23:09:50 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996990, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.32.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 23:09:50 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996990, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.32.84@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
Jun 20 23:09:51 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996990, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.32.84@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
Jun 20 23:09:53 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:09:53.396981 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46115
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:46115
--ranger_config_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:36783/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:09:53.477591 12313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:53.477839 12313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:53.477866 12313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:53.479142 12313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:09:53.479182 12313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:53.479197 12313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:53.479209 12313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:09:53.479249 12313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:09:53.480820 12313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.32.84:36783/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:46115
--ranger_config_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:53.481058 12313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:53.481284 12313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:53.486933 12318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:53.486945 12319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:53.486933 12321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:53.486966 12313 server_base.cc:1034] running on GCE node
I20260620 23:09:53.487411 12313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:53.487653 12313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:53.488811 12313 hybrid_clock.cc:648] HybridClock initialized: now 1781996993488793 us; error 35 us; skew 500 ppm
Jun 20 23:09:53 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996993, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:53.493870 12313 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:09:53.494284 12313 webserver.cc:460] Webserver started at http://127.1.32.126:37771/ using document root <none> and password file <none>
I20260620 23:09:53.494555 12313 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:53.494614 12313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:53.494760 12313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:09:53 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996993, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:09:53.545246 12313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "00f2832850e44620a048f44b6b6f4924"
format_stamp: "Formatted at 2026-06-20 23:09:53 on dist-test-slave-fzjm"
server_key: "280f5efcaafcc2487f78fd8ed53b11fd"
server_key_iv: "852f8707ed4756a11995d488f86c801e"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:53.545655 12313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "00f2832850e44620a048f44b6b6f4924"
format_stamp: "Formatted at 2026-06-20 23:09:53 on dist-test-slave-fzjm"
server_key: "280f5efcaafcc2487f78fd8ed53b11fd"
server_key_iv: "852f8707ed4756a11995d488f86c801e"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:53.547212 12313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:09:53.559875 12328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:53.560073 12313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:09:53.560177 12313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "00f2832850e44620a048f44b6b6f4924"
format_stamp: "Formatted at 2026-06-20 23:09:53 on dist-test-slave-fzjm"
server_key: "280f5efcaafcc2487f78fd8ed53b11fd"
server_key_iv: "852f8707ed4756a11995d488f86c801e"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:53.560262 12313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:53.579507 12313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:53.581007 12313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:53.581156 12313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:53.585940 12313 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:46115
I20260620 23:09:53.585978 12380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:46115 every 8 connection(s)
I20260620 23:09:53.586355 12313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:09:53.587052 12381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:53.589160 12381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924: Bootstrap starting.
I20260620 23:09:53.589648 12381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:53.589871 12381 log.cc:824] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:53.590601 12381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924: No bootstrap required, opened a new log
I20260620 23:09:53.592268 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 12313
I20260620 23:09:53.592373 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:09:53.592557 1153 external_mini_cluster.cc:1428] Setting key 022574d680d6e8625552d7a4ff113bd7
I20260620 23:09:53.592901 12381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f2832850e44620a048f44b6b6f4924" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46115 } }
I20260620 23:09:53.593171 12381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:53.593231 12381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f2832850e44620a048f44b6b6f4924, State: Initialized, Role: FOLLOWER
I20260620 23:09:53.593382 12381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f2832850e44620a048f44b6b6f4924" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46115 } }
I20260620 23:09:53.593463 12381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:09:53.593487 12381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:09:53.593537 12381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:53.594046 12381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f2832850e44620a048f44b6b6f4924" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46115 } }
I20260620 23:09:53.594157 12381 leader_election.cc:302] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00f2832850e44620a048f44b6b6f4924; no voters:
I20260620 23:09:53.594305 12381 leader_election.cc:288] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:09:53.594352 12386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:09:53.594511 12386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [term 1 LEADER]: Becoming Leader. State: Replica: 00f2832850e44620a048f44b6b6f4924, State: Running, Role: LEADER
I20260620 23:09:53.594549 12381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:09:53.594655 12386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f2832850e44620a048f44b6b6f4924" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46115 } }
I20260620 23:09:53.594949 12386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00f2832850e44620a048f44b6b6f4924. Latest consensus state: current_term: 1 leader_uuid: "00f2832850e44620a048f44b6b6f4924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f2832850e44620a048f44b6b6f4924" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46115 } } }
I20260620 23:09:53.594959 12388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00f2832850e44620a048f44b6b6f4924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f2832850e44620a048f44b6b6f4924" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46115 } } }
I20260620 23:09:53.595032 12386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [sys.catalog]: This master's current role is: LEADER
I20260620 23:09:53.595049 12388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924 [sys.catalog]: This master's current role is: LEADER
I20260620 23:09:53.596043 12381 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:09:53.596422 12392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:09:53.599366 12392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:09:53.601787 12392 catalog_manager.cc:1347] Generated new cluster ID: a3797642e77c4339a0486781cb47813a
I20260620 23:09:53.601846 12392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:09:53 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:09:53.605865 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:53.593132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.1.32.126:46115)
0620 23:09:53.593293 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:53.593302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:53.593332 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:09:53.593438 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:53.593443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:53.593511 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:53.593654 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:53.594288 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:53.595663 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:53.597889 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:53.599353 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:53.599508 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:53.602507 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:53.602511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:53.602511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:53.602522 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:53.604388 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:53.604737 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:53.604740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:53.604740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:53.604855 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:53.605039 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:53.605040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:53.605041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:53.605307 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:53.605451 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:53.605511 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 23:09:53.605635 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260620 23:09:53.619524 12392 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:09:53.620239 12392 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:09:53.628726 12392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924: Generated new TSK 0
I20260620 23:09:53.628886 12392 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:09:54.284090 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:36783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46115
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:09:54.398109 12426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:54.398324 12426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:54.398358 12426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:54.399471 12426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:54.399528 12426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:54.399567 12426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:09:54.401024 12426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.32.84:36783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:54.401247 12426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:54.401515 12426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:54.407431 12434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:54.407436 12431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:54.407603 12426 server_base.cc:1034] running on GCE node
W20260620 23:09:54.407737 12432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:54.407933 12426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:54.408216 12426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:54.409366 12426 hybrid_clock.cc:648] HybridClock initialized: now 1781996994409352 us; error 30 us; skew 500 ppm
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:54.415822 12426 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:09:54.416338 12426 webserver.cc:460] Webserver started at http://127.1.32.65:35645/ using document root <none> and password file <none>
I20260620 23:09:54.416610 12426 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:54.416678 12426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:54.416807 12426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:09:54.436444 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "c0b5da1eae0043d7b2f67b30cb10d473"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "1f22fc337fa9f7be9df58f1e6af2d747"
server_key_iv: "dcb6ebae3196a81b51b4dbe0173f89f6"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.436836 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "c0b5da1eae0043d7b2f67b30cb10d473"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "1f22fc337fa9f7be9df58f1e6af2d747"
server_key_iv: "dcb6ebae3196a81b51b4dbe0173f89f6"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.438613 12426 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260620 23:09:54.454766 12442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:54.460687 12426 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260620 23:09:54.460803 12426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "c0b5da1eae0043d7b2f67b30cb10d473"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "1f22fc337fa9f7be9df58f1e6af2d747"
server_key_iv: "dcb6ebae3196a81b51b4dbe0173f89f6"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.460891 12426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:54.492025 12426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:54.494825 12426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:54.495076 12426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:54.495330 12426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:54.495682 12426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:54.495733 12426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:54.495762 12426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:54.495779 12426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:54.504635 12426 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:46401
I20260620 23:09:54.505107 12426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:09:54.505713 12556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:46401 every 8 connection(s)
I20260620 23:09:54.510648 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 12426
I20260620 23:09:54.510749 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:09:54.510994 1153 external_mini_cluster.cc:1428] Setting key 3508d6195583dd94b7dfa53440d8fd6d
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
I20260620 23:09:54.525552 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:54.506952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46115 (local address 127.1.32.65:59159)
0620 23:09:54.507108 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:54.507149 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:54.507601 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:54.507922 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:54.507927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:54.508110 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:54.508297 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:54.508301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.509093 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.509095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:54.509907 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:54.509909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.510344 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.510345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:54.510522 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:54.511169 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:54.511178 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:54.512842 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:54.523875 (+ 11033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:54.523883 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:54.523885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:54.524206 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:54.524447 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:54.524448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:54.524448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:54.524578 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:54.524871 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:54.524878 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:54.525056 (+ 178us) client_negotiation.cc:769] Sending connection context
0620 23:09:54.525159 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 23:09:54.525314 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260620 23:09:54.526041 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:54.506994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:59159 (local address 127.1.32.126:46115)
0620 23:09:54.507730 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:54.507732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:54.507738 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:09:54.507759 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:54.507760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:54.507769 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:54.507849 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:54.508424 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.508980 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.510024 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.510229 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.510325 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:54.512954 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:54.512955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:54.512956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:54.512964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:54.514962 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:54.524303 (+ 9341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:54.524304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:54.524304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:54.524375 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:54.524649 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:54.524650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:54.524650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:54.524793 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:54.525802 (+ 1009us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:54.525844 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 23:09:54.525946 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":84,"threads_started":1}
I20260620 23:09:54.526386 12557 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46115
I20260620 23:09:54.526487 12557 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:54.526700 12557 heartbeater.cc:511] Master 127.1.32.126:46115 requested a full tablet report, sending...
I20260620 23:09:54.527241 12337 ts_manager.cc:194] Registered new tserver with Master: c0b5da1eae0043d7b2f67b30cb10d473 (127.1.32.65:46401)
I20260620 23:09:54.528124 12337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:59159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:09:54.542059 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:36783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46115
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 23:09:54.664113 12563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:54.664459 12563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:54.664505 12563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:54.666534 12563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:54.666617 12563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:54.666682 12563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:09:54.668586 12563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.32.84:36783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:54.668905 12563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:54.669142 12563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:54.675431 12570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:54.675719 12573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:54.675851 12571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:54.679473 12563 server_base.cc:1034] running on GCE node
I20260620 23:09:54.679797 12563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:54.680099 12563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:54.681283 12563 hybrid_clock.cc:648] HybridClock initialized: now 1781996994681263 us; error 36 us; skew 500 ppm
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:54.684720 12563 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:09:54.685297 12563 webserver.cc:460] Webserver started at http://127.1.32.66:40891/ using document root <none> and password file <none>
I20260620 23:09:54.685601 12563 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:54.685745 12563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:54.685948 12563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:09:54.704980 12563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "ff5b725f826a4e8fa36577d577886b8a"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "3c54e03b00e7307fde98e73cba5c1321"
server_key_iv: "d0b785b3519eeec6425c336039aecba0"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.705471 12563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "ff5b725f826a4e8fa36577d577886b8a"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "3c54e03b00e7307fde98e73cba5c1321"
server_key_iv: "d0b785b3519eeec6425c336039aecba0"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.710829 12563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260620 23:09:54.724133 12581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:54.724398 12563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:09:54.724581 12563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "ff5b725f826a4e8fa36577d577886b8a"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "3c54e03b00e7307fde98e73cba5c1321"
server_key_iv: "d0b785b3519eeec6425c336039aecba0"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.724782 12563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:54.747489 12563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:54.749894 12563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:54.750108 12563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:54.750392 12563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:54.750780 12563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:54.750881 12563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:54.750993 12563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:54.751058 12563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:54.758143 12563 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:41831
I20260620 23:09:54.761344 12694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:41831 every 8 connection(s)
I20260620 23:09:54.761723 12563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:09:54.769711 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 12563
I20260620 23:09:54.769809 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:09:54.769979 1153 external_mini_cluster.cc:1428] Setting key 167eca112acd1a55f4b2cd169076390b
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:09:54.777402 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:54.763012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:49627 (local address 127.1.32.126:46115)
0620 23:09:54.763070 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:54.763072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:54.767209 (+ 4137us) server_negotiation.cc:407] Connection header received
0620 23:09:54.767921 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:54.767923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:54.767936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:54.767968 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:54.768808 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.769602 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.771116 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.771443 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.771487 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:54.774073 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:54.774074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:54.774075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:54.774082 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:54.775872 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:54.776326 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:54.776327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:54.776328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:54.776382 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:54.776637 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:54.776638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:54.776639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:54.776766 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:54.776822 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:54.777213 (+ 391us) server_negotiation.cc:299] Negotiation successful
0620 23:09:54.777295 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:09:54.778312 12697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:54.762910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46115 (local address 127.1.32.66:49627)
0620 23:09:54.767055 (+ 4145us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:54.767107 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:54.767779 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:54.768160 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:54.768164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:54.768409 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:54.768683 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:54.768689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.769721 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.769722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:54.771022 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:54.771025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:54.771580 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:54.771581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:54.771732 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:54.772312 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:54.772322 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:54.773967 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:54.775963 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:54.775969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:54.775972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:54.776239 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:54.776448 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:54.776448 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:54.776449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:54.776567 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:54.776871 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:54.776877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:54.777072 (+ 195us) client_negotiation.cc:769] Sending connection context
0620 23:09:54.777921 (+ 849us) client_negotiation.cc:240] Negotiation successful
0620 23:09:54.778090 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4095,"thread_start_us":83,"threads_started":1}
I20260620 23:09:54.778951 12695 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46115
I20260620 23:09:54.779039 12695 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:54.779211 12695 heartbeater.cc:511] Master 127.1.32.126:46115 requested a full tablet report, sending...
I20260620 23:09:54.779603 12337 ts_manager.cc:194] Registered new tserver with Master: ff5b725f826a4e8fa36577d577886b8a (127.1.32.66:41831)
I20260620 23:09:54.780061 12337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:49627
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:09:54.812330 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:36783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46115
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:09:54.934657 12700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:09:54.934942 12700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:09:54.934975 12700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:09:54.936705 12700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:09:54.936772 12700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:09:54.936810 12700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:09:54.938777 12700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.32.84:36783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:09:54.939054 12700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:09:54.939342 12700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:09:54.945772 12706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:54.945904 12708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:09:54.945782 12705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:09:54.946956 12700 server_base.cc:1034] running on GCE node
I20260620 23:09:54.947221 12700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:09:54.947507 12700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:09:54.948668 12700 hybrid_clock.cc:648] HybridClock initialized: now 1781996994948642 us; error 36 us; skew 500 ppm
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:09:54.952668 12700 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:09:54.953178 12700 webserver.cc:460] Webserver started at http://127.1.32.67:35645/ using document root <none> and password file <none>
I20260620 23:09:54.953449 12700 fs_manager.cc:359] Metadata directory not provided
I20260620 23:09:54.953509 12700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:09:54.953686 12700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:09:54 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:09:54.977840 12700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "602ee5230c0e46ddb1816c389406137c"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "3956958ae8f115e82c8f891bbd86a693"
server_key_iv: "54ddc647375493cfa5a5356803327738"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.978322 12700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "602ee5230c0e46ddb1816c389406137c"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "3956958ae8f115e82c8f891bbd86a693"
server_key_iv: "54ddc647375493cfa5a5356803327738"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:54.986114 12700 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260620 23:09:55.003132 12715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:55.003428 12700 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260620 23:09:55.003513 12700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "602ee5230c0e46ddb1816c389406137c"
format_stamp: "Formatted at 2026-06-20 23:09:54 on dist-test-slave-fzjm"
server_key: "3956958ae8f115e82c8f891bbd86a693"
server_key_iv: "54ddc647375493cfa5a5356803327738"
server_key_version: "kuduclusterkey@0"
I20260620 23:09:55.003604 12700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:09:55.017771 12700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:09:55.021881 12700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:09:55.022070 12700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:09:55.022329 12700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:09:55.022686 12700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:09:55.022776 12700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:55.022840 12700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:09:55.022913 12700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:09:55.029220 12700 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:41971
I20260620 23:09:55.029624 12700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:09:55.031752 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 12700
I20260620 23:09:55.031858 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:09:55.032056 1153 external_mini_cluster.cc:1428] Setting key 137cbfa0c2db3fc206a5a33197ac8cb9
I20260620 23:09:55.032708 12828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:41971 every 8 connection(s)
Jun 20 23:09:55 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:09:55.052769 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.033507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:59631 (local address 127.1.32.126:46115)
0620 23:09:55.033544 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.033546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.043226 (+ 9680us) server_negotiation.cc:407] Connection header received
0620 23:09:55.044016 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.044017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.044028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.044069 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:55.044592 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.045335 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.046748 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.047089 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.047133 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.049550 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:55.049551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:55.049551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:55.049558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:55.051260 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:55.051733 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:55.051734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:55.051735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:55.051803 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:55.052051 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:55.052052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:55.052052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:55.052173 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:55.052226 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.052585 (+ 359us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.052670 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:09:55.053505 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.033250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:46115 (local address 127.1.32.67:59631)
0620 23:09:55.042985 (+ 9735us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.043048 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:55.043907 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:55.044136 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:55.044139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:55.044319 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:09:55.044505 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.044509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.045445 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.045446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.046651 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.046653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.047230 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.047231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.047367 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.047911 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:09:55.047921 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:09:55.049453 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:09:55.051351 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:55.051357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:55.051359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:55.051649 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:55.051867 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:09:55.051868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:09:55.051868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:09:55.051981 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:09:55.052271 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:09:55.052276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:09:55.052442 (+ 166us) client_negotiation.cc:769] Sending connection context
0620 23:09:55.053112 (+ 670us) client_negotiation.cc:240] Negotiation successful
0620 23:09:55.053280 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9681,"thread_start_us":69,"threads_started":1}
I20260620 23:09:55.054384 12829 heartbeater.cc:347] Connected to a master server at 127.1.32.126:46115
I20260620 23:09:55.054737 12829 heartbeater.cc:464] Registering TS with master...
I20260620 23:09:55.054987 12829 heartbeater.cc:511] Master 127.1.32.126:46115 requested a full tablet report, sending...
I20260620 23:09:55.055382 12337 ts_manager.cc:194] Registered new tserver with Master: 602ee5230c0e46ddb1816c389406137c (127.1.32.67:41971)
I20260620 23:09:55.056895 12337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:59631
I20260620 23:09:55.065240 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:09:55.078454 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.067212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.1.32.126:46115)
0620 23:09:55.067251 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.067253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.071075 (+ 3822us) server_negotiation.cc:407] Connection header received
0620 23:09:55.071173 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.071174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.071193 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.071221 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:09:55.071955 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.072770 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.073786 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.074066 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.074102 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.075648 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:09:55.075649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:09:55.075649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:09:55.075656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:09:55.077317 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:55.077677 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:55.077677 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:55.077678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:55.077730 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:09:55.077905 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:09:55.077906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:09:55.077906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:09:55.078012 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:09:55.078080 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.078296 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.078365 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:09:55.080564 12337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:09:55.082669 12337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:09:55.102999 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.093399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46401 (local address 127.0.0.1:37880)
0620 23:09:55.094032 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.094039 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:55.094066 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:55.094776 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:55.094779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:55.094781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:55.094858 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.094860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.099956 (+ 5096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.099957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.101573 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.101577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.102645 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.102646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.102722 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.102736 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 23:09:55.102808 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 23:09:55.102871 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":67,"threads_started":1}
I20260620 23:09:55.108268 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.093488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.1.32.65:46401)
0620 23:09:55.094543 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.094547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.094556 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:09:55.094587 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.094589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.094603 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.094669 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:55.095880 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.096904 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.101691 (+ 4787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.102471 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.102545 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.102608 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.108136 (+ 5528us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.108188 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":51,"threads_started":1}
I20260620 23:09:55.108762 12845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.092937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:41971 (local address 127.0.0.1:57098)
0620 23:09:55.093740 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.093779 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:55.093848 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:55.094987 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:55.094988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:55.094990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:55.095021 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.095022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.103479 (+ 8457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.103480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.104897 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.104899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.108550 (+ 3651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.108551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.108615 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.108620 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:09:55.108652 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 23:09:55.108687 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":70,"threads_started":1}
I20260620 23:09:55.109367 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.093342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.1.32.67:41971)
0620 23:09:55.094363 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.094367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.094375 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:09:55.094410 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.094412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.094426 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.095185 (+ 759us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:55.095263 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.099838 (+ 4575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.105021 (+ 5183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.105795 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.109122 (+ 3327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.109218 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.109245 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.109289 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":87,"threads_started":1}
I20260620 23:09:55.110176 12491 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51 (DEFAULT_TABLE table=test-table [id=9cbbfa40e2a046b780d35e6988351b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:09:55.110564 12491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:55.114509 12847 tablet_bootstrap.cc:492] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473: Bootstrap starting.
I20260620 23:09:55.115272 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.093146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41831 (local address 127.0.0.1:42302)
0620 23:09:55.093940 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.093947 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:55.093975 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:55.095092 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:55.095092 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:55.095094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:55.095118 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.095119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.105919 (+ 10800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.105920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.107005 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.107007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.115004 (+ 7997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.115005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.115091 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.115096 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:09:55.115125 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 23:09:55.115164 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":56,"threads_started":1}
I20260620 23:09:55.115342 12847 tablet_bootstrap.cc:654] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:55.115895 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.093252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.1.32.66:41831)
0620 23:09:55.094169 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.094174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.094183 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:09:55.094219 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.094221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.094235 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.095567 (+ 1332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:55.095642 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.099235 (+ 3593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.107125 (+ 7890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.114881 (+ 7756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.115651 (+ 770us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.115738 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.115765 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.115806 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":83,"threads_started":1}
I20260620 23:09:55.116326 12847 log.cc:824] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:55.116667 12629 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51 (DEFAULT_TABLE table=test-table [id=9cbbfa40e2a046b780d35e6988351b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:09:55.116999 12629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:55.117115 12847 tablet_bootstrap.cc:492] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473: No bootstrap required, opened a new log
I20260620 23:09:55.117163 12847 ts_tablet_manager.cc:1397] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260620 23:09:55.119650 12847 raft_consensus.cc:348] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.119807 12847 raft_consensus.cc:374] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:55.119854 12847 raft_consensus.cc:729] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b5da1eae0043d7b2f67b30cb10d473, State: Initialized, Role: FOLLOWER
I20260620 23:09:55.120272 12847 consensus_queue.cc:260] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.120545 12847 ts_tablet_manager.cc:1428] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 23:09:55.120704 12557 heartbeater.cc:503] Master 127.1.32.126:46115 was elected leader, sending a full tablet report...
I20260620 23:09:55.122080 12849 tablet_bootstrap.cc:492] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: Bootstrap starting.
I20260620 23:09:55.123502 12849 tablet_bootstrap.cc:654] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:55.123857 12849 log.cc:824] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:55.124603 12849 tablet_bootstrap.cc:492] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: No bootstrap required, opened a new log
I20260620 23:09:55.124668 12849 ts_tablet_manager.cc:1397] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 23:09:55.124750 12763 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51 (DEFAULT_TABLE table=test-table [id=9cbbfa40e2a046b780d35e6988351b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:09:55.125057 12763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:09:55.126526 12849 raft_consensus.cc:348] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.126682 12849 raft_consensus.cc:374] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:55.126713 12849 raft_consensus.cc:729] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff5b725f826a4e8fa36577d577886b8a, State: Initialized, Role: FOLLOWER
I20260620 23:09:55.127009 12849 consensus_queue.cc:260] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.127283 12849 ts_tablet_manager.cc:1428] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:09:55.127389 12695 heartbeater.cc:503] Master 127.1.32.126:46115 was elected leader, sending a full tablet report...
I20260620 23:09:55.128414 12853 tablet_bootstrap.cc:492] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Bootstrap starting.
I20260620 23:09:55.129398 12853 tablet_bootstrap.cc:654] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Neither blocks nor log segments found. Creating new log.
I20260620 23:09:55.129769 12853 log.cc:824] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:09:55.145944 12853 tablet_bootstrap.cc:492] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: No bootstrap required, opened a new log
I20260620 23:09:55.146036 12853 ts_tablet_manager.cc:1397] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20260620 23:09:55.148577 12853 raft_consensus.cc:348] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.148836 12853 raft_consensus.cc:374] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:09:55.148910 12853 raft_consensus.cc:729] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602ee5230c0e46ddb1816c389406137c, State: Initialized, Role: FOLLOWER
I20260620 23:09:55.149084 12853 consensus_queue.cc:260] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.149371 12853 ts_tablet_manager.cc:1428] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:09:55.149407 12829 heartbeater.cc:503] Master 127.1.32.126:46115 was elected leader, sending a full tablet report...
Jun 20 23:09:55 dist-test-slave-fzjm krb5kdc[11869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996994, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
W20260620 23:09:55.256690 12558 tablet.cc:2369] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:09:55.262684 12696 tablet.cc:2369] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:09:55.283331 12830 tablet.cc:2369] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:09:55.311236 12852 raft_consensus.cc:482] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:09:55.311381 12852 raft_consensus.cc:504] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.311780 12852 leader_election.cc:288] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602ee5230c0e46ddb1816c389406137c (127.1.32.67:41971), c0b5da1eae0043d7b2f67b30cb10d473 (127.1.32.65:46401)
I20260620 23:09:55.319007 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.312536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:41045 (local address 127.1.32.67:41971)
0620 23:09:55.312607 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.312610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.312885 (+ 275us) server_negotiation.cc:407] Connection header received
0620 23:09:55.313010 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.313012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.313026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.313060 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:55.313296 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.314125 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.317991 (+ 3866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.318548 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.318599 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.318610 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.318869 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.318929 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:09:55.319504 12857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.312036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:41971 (local address 127.1.32.66:41045)
0620 23:09:55.312795 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.312810 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:55.312920 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:55.313138 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:55.313139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:55.313141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:55.313211 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.313214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.314268 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.314269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.315560 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.315563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.318689 (+ 3126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.318690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.318779 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.318783 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:09:55.319351 (+ 568us) client_negotiation.cc:240] Negotiation successful
0620 23:09:55.319401 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":71,"threads_started":1}
I20260620 23:09:55.320938 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.312473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:55365 (local address 127.1.32.65:46401)
0620 23:09:55.312536 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.312538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.315752 (+ 3214us) server_negotiation.cc:407] Connection header received
0620 23:09:55.315856 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.315857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.315869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.315899 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:09:55.316097 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.316733 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.319751 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.320309 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.320356 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.320365 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.320810 (+ 445us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.320854 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 23:09:55.320868 12783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd883b9d0e74dd4a4d1dea26fffdf51" candidate_uuid: "ff5b725f826a4e8fa36577d577886b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602ee5230c0e46ddb1816c389406137c" is_pre_election: true
I20260620 23:09:55.321058 12783 raft_consensus.cc:2454] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff5b725f826a4e8fa36577d577886b8a in term 0.
I20260620 23:09:55.321314 12584 leader_election.cc:302] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602ee5230c0e46ddb1816c389406137c, ff5b725f826a4e8fa36577d577886b8a; no voters:
I20260620 23:09:55.321475 12852 raft_consensus.cc:2781] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:09:55.321529 12852 raft_consensus.cc:482] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:09:55.321563 12852 raft_consensus.cc:3037] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:55.322467 12852 raft_consensus.cc:504] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.322629 12852 leader_election.cc:288] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [CANDIDATE]: Term 1 election: Requested vote from peers 602ee5230c0e46ddb1816c389406137c (127.1.32.67:41971), c0b5da1eae0043d7b2f67b30cb10d473 (127.1.32.65:46401)
I20260620 23:09:55.322841 12783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd883b9d0e74dd4a4d1dea26fffdf51" candidate_uuid: "ff5b725f826a4e8fa36577d577886b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602ee5230c0e46ddb1816c389406137c"
I20260620 23:09:55.322956 12783 raft_consensus.cc:3037] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:55.323855 12783 raft_consensus.cc:2454] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff5b725f826a4e8fa36577d577886b8a in term 1.
I20260620 23:09:55.324034 12584 leader_election.cc:302] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602ee5230c0e46ddb1816c389406137c, ff5b725f826a4e8fa36577d577886b8a; no voters:
I20260620 23:09:55.324131 12852 raft_consensus.cc:2781] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:09:55.324299 12852 raft_consensus.cc:686] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 1 LEADER]: Becoming Leader. State: Replica: ff5b725f826a4e8fa36577d577886b8a, State: Running, Role: LEADER
I20260620 23:09:55.324404 12852 consensus_queue.cc:237] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } }
I20260620 23:09:55.325244 12337 catalog_manager.cc:5495] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to ff5b725f826a4e8fa36577d577886b8a (127.1.32.66). New cstate: current_term: 1 leader_uuid: "ff5b725f826a4e8fa36577d577886b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff5b725f826a4e8fa36577d577886b8a" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41831 } health_report { overall_health: HEALTHY } } }
I20260620 23:09:55.325740 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.312230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46401 (local address 127.1.32.66:55365)
0620 23:09:55.315683 (+ 3453us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.315694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:09:55.315780 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:09:55.315959 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:09:55.315960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:09:55.315961 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:09:55.316025 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.316027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.316839 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.316840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.317883 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:09:55.317885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.320426 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.320426 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:09:55.320492 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.320495 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:09:55.321258 (+ 763us) client_negotiation.cc:240] Negotiation successful
0620 23:09:55.321301 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3438,"thread_start_us":72,"threads_started":1}
I20260620 23:09:55.326172 12511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd883b9d0e74dd4a4d1dea26fffdf51" candidate_uuid: "ff5b725f826a4e8fa36577d577886b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" is_pre_election: true
I20260620 23:09:55.326326 12511 raft_consensus.cc:2454] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff5b725f826a4e8fa36577d577886b8a in term 0.
I20260620 23:09:55.326474 12510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd883b9d0e74dd4a4d1dea26fffdf51" candidate_uuid: "ff5b725f826a4e8fa36577d577886b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b5da1eae0043d7b2f67b30cb10d473"
I20260620 23:09:55.326539 12510 raft_consensus.cc:3037] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:09:55.327378 12510 raft_consensus.cc:2454] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff5b725f826a4e8fa36577d577886b8a in term 1.
I20260620 23:09:55.380728 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.377364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.1.32.66:41831)
0620 23:09:55.377408 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:09:55.377410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:09:55.377419 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:09:55.377492 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:09:55.377492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:09:55.377513 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:09:55.377591 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:09:55.377764 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.378475 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.379662 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:09:55.380045 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:09:55.380121 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:09:55.380315 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:09:55.380445 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:09:55.380536 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 23:09:55.380590 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 23:09:55.380615 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:09:55.382781 12510 raft_consensus.cc:1261] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P c0b5da1eae0043d7b2f67b30cb10d473 [term 1 FOLLOWER]: Refusing update from remote peer ff5b725f826a4e8fa36577d577886b8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:09:55.383090 12783 raft_consensus.cc:1261] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 1 FOLLOWER]: Refusing update from remote peer ff5b725f826a4e8fa36577d577886b8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:09:55.383483 12852 consensus_queue.cc:1035] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "602ee5230c0e46ddb1816c389406137c" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:09:55.383688 12858 consensus_queue.cc:1035] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0b5da1eae0043d7b2f67b30cb10d473" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:09:55.387882 12861 mvcc.cc:205] Tried to move back new op lower bound from 7299059693085032448 to 7299059692849401856. Current Snapshot: MvccSnapshot[applied={T|T < 7299059693085032448}]
I20260620 23:09:55.389138 12864 mvcc.cc:205] Tried to move back new op lower bound from 7299059693085032448 to 7299059692849401856. Current Snapshot: MvccSnapshot[applied={T|T < 7299059693085032448}]
I20260620 23:09:55.390434 12337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:09:55.390522 12337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:09:55.391278 12337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00f2832850e44620a048f44b6b6f4924: Sending DeleteTablet for 3 replicas of tablet 6bd883b9d0e74dd4a4d1dea26fffdf51
I20260620 23:09:55.391741 12491 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:09:55 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:37880
I20260620 23:09:55.391770 12629 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:09:55 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:42302
I20260620 23:09:55.391955 12763 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:09:55 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:57098
I20260620 23:09:55.392493 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 12426
I20260620 23:09:55.394070 12871 tablet_replica.cc:330] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: stopping tablet replica
I20260620 23:09:55.394243 12871 raft_consensus.cc:2229] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:09:55.394438 12871 pending_rounds.cc:62] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Trying to abort 2 pending ops.
I20260620 23:09:55.394569 12871 pending_rounds.cc:68] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7299059693085032448 op_type: WRITE_OP write_request { tablet_id: "6bd883b9d0e74dd4a4d1dea26fffdf51" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\357\275\334\321\006\"8\n\ntest-admin\022*\n 9cbbfa40e2a046b780d35e6988351b24\020\001\030\001 \001(\001" signature: "<redacted>""\023U@C\207\211M\213\232\3377\302.u\"\"Pv\200\331\356\r\346>f7\246\034@L\3545L\372\277\255\274\234|\035\t\013L\337F\367\007\0253\034\325\275\275\025\324\373\223E\324~\357/`\020" signing_key_seq_num: 0 } } request_id { client_id: "4cfe51ebf93a4ed7b5f5aaf3ef144e3b" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
W20260620 23:09:55.394950 12863 consensus_queue.cc:1159] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "602ee5230c0e46ddb1816c389406137c" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260620 23:09:55.394976 12871 pending_rounds.cc:68] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7299059692849401856 op_type: NO_OP noop_request { }
I20260620 23:09:55.395553 12871 raft_consensus.cc:2866] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260620 23:09:55.395720 12871 raft_consensus.cc:2258] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 23:09:55.395920 12584 consensus_peers.cc:487] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a -> Peer 602ee5230c0e46ddb1816c389406137c (127.1.32.67:41971): Couldn't send request to peer 602ee5230c0e46ddb1816c389406137c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260620 23:09:55.399864 12871 ts_tablet_manager.cc:1905] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:09:55.401684 12870 tablet_replica.cc:330] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: stopping tablet replica
I20260620 23:09:55.401837 12870 raft_consensus.cc:2229] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:09:55.402004 12870 raft_consensus.cc:2258] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 23:09:55.402323 12329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:46401 (error 108)
I20260620 23:09:55.402508 12870 ts_tablet_manager.cc:1905] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P ff5b725f826a4e8fa36577d577886b8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:09:55.403164 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 12563
I20260620 23:09:55.403177 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.403047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46401 (local address 127.0.0.1:37894)
0620 23:09:55.403093 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.403143 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260620 23:09:55.403285 12329 catalog_manager.cc:4584] TS c0b5da1eae0043d7b2f67b30cb10d473 (127.1.32.65:46401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6bd883b9d0e74dd4a4d1dea26fffdf51: Network error: Client connection negotiation failed: client connection to 127.1.32.65:46401: connect: Connection refused (error 111)
I20260620 23:09:55.403828 12871 ts_tablet_manager.cc:1918] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:09:55.403935 12871 log.cc:1196] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/6bd883b9d0e74dd4a4d1dea26fffdf51
I20260620 23:09:55.404520 12871 ts_tablet_manager.cc:1939] T 6bd883b9d0e74dd4a4d1dea26fffdf51 P 602ee5230c0e46ddb1816c389406137c: Deleting consensus metadata
I20260620 23:09:55.405019 12331 catalog_manager.cc:4841] TS 602ee5230c0e46ddb1816c389406137c (127.1.32.67:41971): tablet 6bd883b9d0e74dd4a4d1dea26fffdf51 (table test-table [id=9cbbfa40e2a046b780d35e6988351b24]) successfully deleted
I20260620 23:09:55.408664 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 12700
I20260620 23:09:55.408811 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:09:55.408716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41831 (local address 127.0.0.1:42326)
0620 23:09:55.408764 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:09:55.408786 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:41831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 23:09:55.414423 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 12313
I20260620 23:09:55.419824 1153 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 23:09:55.753799 1153 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 23:09:55.753912 1153 mini_ranger.cc:66] Stopping Ranger...
I20260620 23:09:56.097663 1153 mini_ranger.cc:68] Stopped Ranger
2026-06-20 23:09:56.097 UTC [11899] LOG: received smart shutdown request
2026-06-20 23:09:56.101 UTC [11899] LOG: background worker "logical replication launcher" (PID 11908) exited with exit code 1
2026-06-20 23:09:56.101 UTC [11903] LOG: shutting down
2026-06-20 23:09:56.245 UTC [11899] LOG: database system is shut down
2026-06-20T23:09:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43699 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:09:56 dist-test-slave-fzjm krb5kdc[12881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:09:56 dist-test-slave-fzjm krb5kdc[12881](info): set up 2 sockets
Jun 20 23:09:56 dist-test-slave-fzjm krb5kdc[12881](info): commencing operation
krb5kdc: starting...
W20260620 23:09:57.362603 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:09:57 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:09:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:09:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/rangeradmin_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/rangeradmin_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/rangerlookup_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/rangerlookup_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
I20260620 23:09:57.510959 1153 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfTMrFo/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/postgres -l logfile start
2026-06-20 23:09:59.746 UTC [12911] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 23:09:59.746 UTC [12911] LOG: listening on IPv4 address "127.1.32.84", port 45177
2026-06-20 23:09:59.749 UTC [12911] LOG: listening on Unix socket "/tmp/.s.PGSQL.45177"
2026-06-20 23:09:59.767 UTC [12914] LOG: database system was shut down at 2026-06-20 23:09:58 UTC
2026-06-20 23:09:59.770 UTC [12911] LOG: database system is ready to accept connections
I20260620 23:10:00.690402 1153 mini_postgres.cc:96] Postgres bound to 45177
2026-06-20 23:10:00.696 UTC [12922] FATAL: database "slave" does not exist
127.1.32.84:45177 - accepting connections
I20260620 23:10:00.696484 1153 mini_ranger.cc:161] Starting Ranger...
I20260620 23:10:00.710266 1153 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 23:10:01.125923 1153 mini_ranger.cc:87] Created ranger Postgres database
I20260620 23:10:01.125991 1153 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-admin
2026-06-20 23:10:01,400 [I] DB FLAVOR :POSTGRES
2026-06-20 23:10:01,401 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:10:01,401 [I] Checking connection..
2026-06-20 23:10:01,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 23:10:01,645 [I] Checking connection passed.
2026-06-20 23:10:01,646 [I] --------- Verifying version history table ---------
2026-06-20 23:10:01,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:10:01,889 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:10:01,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:10:02,132 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:10:02,133 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 23:10:02,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 23:10:02.371 UTC [12986] WARNING: there is no transaction in progress
2026-06-20 23:10:02,378 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 23:10:02,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:10:02,601 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 23:10:02,601 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:10:02,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 23:10:02,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 23:10:03,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fzjm.c.gcp-upstream.internal','N') ;"
2026-06-20 23:10:03,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:10:03,535 [I] Table x_portal_user does not exist in database ranger
2026-06-20 23:10:03,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:10:03,772 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 23:10:03,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:10:04,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:10:04,234 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 23:10:04,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 23:10:05.146 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.161 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.175 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.189 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.203 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.219 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.274 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.281 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.293 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.302 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.311 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.325 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.334 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.340 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.347 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05.353 UTC [13094] WARNING: there is no transaction in progress
2026-06-20 23:10:05,926 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 23:10:05,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 23:10:06,175 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 23:10:06,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:10:06,419 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 23:10:06,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:10:06,660 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 23:10:06,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:10:06,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:10:07,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fzjm.c.gcp-upstream.internal';"
2026-06-20 23:10:07,347 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 23:10:07,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 23:10:07,569 [I] DB_PATCHES have already been applied
I20260620 23:10:07.574672 1153 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfTMrFo/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 23:10:07.574769 1153 mini_ranger.cc:193] Using host: 127.1.32.84
I20260620 23:10:07.577675 1153 mini_ranger.cc:227] Ranger admin URL: http://127.1.32.84:45827
Jun 20, 2026 11:10:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:10:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:10:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:10:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:10:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 23:10:08.676532 1153 mini_ranger.cc:160] Time spent starting Ranger: real 7.980s user 0.000s sys 0.008s
Jun 20, 2026 11:10:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:10:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 11:10:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:10:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:10:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 23:10:29.973860 1153 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.32.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.32.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/rangerkms_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.32.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/rangerkms_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.32.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.32.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.32.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/HTTP_127.1.32.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 23:10:30 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997030, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 23:10:30.051981 1153 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 23:10:30.060626 1153 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 23:10:30.798385 1153 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 23:10:30.798463 1153 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms
I20260620 23:10:30.798483 1153 mini_ranger_kms.cc:227] Using postgres at 127.1.32.84:45177
2026-06-20 23:10:30,955 [I] DB FLAVOR :POSTGRES
2026-06-20 23:10:30,956 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:10:30,956 [I] Checking connection
2026-06-20 23:10:30,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 23:10:31,173 [I] connection success
2026-06-20 23:10:31,173 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 23:10:31,174 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 23:10:31,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 23:10:31,406 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 23:10:31,406 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:10:31,406 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 23:10:31,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.32.84:45177/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 23:10:31,675 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 23:10:32.068168 1153 mini_ranger_kms.cc:310] Created kms service
I20260620 23:10:32.221618 1153 mini_ranger_kms.cc:326] Created kudu user
I20260620 23:10:32.261580 1153 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 23:10:32 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997030, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:32.601248 1153 mini_ranger_kms.cc:384] Added ranger policy
I20260620 23:10:32.601353 1153 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfTMrFo/build/release/bin/postgresql.jar:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfTMrFo/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 23:10:32.601372 1153 mini_ranger_kms.cc:240] Using host: 127.1.32.84
I20260620 23:10:32.605283 1153 mini_ranger_kms.cc:276] Ranger KMS PID: 13274
I20260620 23:10:32.605366 1153 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.32.84:53079
Jun 20, 2026 11:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:10:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:10:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 23:10:34.754256 1153 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.702s user 0.000s sys 0.141s
I20260620 23:10:34.754354 1153 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 23:10:34.754402 1153 mini_ranger_kms.cc:401] 127.1.32.84:53079/kms/v1/keys
Jun 20 23:10:34 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781997034, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.32.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 23:10:35 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997034, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.32.84@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
Jun 20 23:10:35 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997034, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.32.84@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
Jun 20 23:10:37 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:10:37.913930 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:37469
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:37469
--ranger_config_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:37.989284 13325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:37.989445 13325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:37.989464 13325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:37.990543 13325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:37.990581 13325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:37.990594 13325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:37.990607 13325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:37.990618 13325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:37.992132 13325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.32.84:53079/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:37469
--ranger_config_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:37469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:37.992342 13325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:37.992559 13325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:37.997560 13331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:37.997586 13330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:37.997670 13333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:37.997700 13325 server_base.cc:1034] running on GCE node
I20260620 23:10:37.998000 13325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:37.998248 13325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:37.999396 13325 hybrid_clock.cc:648] HybridClock initialized: now 1781997037999369 us; error 41 us; skew 500 ppm
Jun 20 23:10:37 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997037, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:38.001681 13325 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:10:38.002132 13325 webserver.cc:460] Webserver started at http://127.1.32.126:42829/ using document root <none> and password file <none>
I20260620 23:10:38.002360 13325 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:38.002398 13325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:38.002509 13325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:10:37 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997037, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:38.056094 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "f9f50f86382941d7a810b00eb88fe3ac"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "cd1845135d60a667fb77fc0ffac427a9"
server_key_iv: "28449b91bf35795a97b3bec084252f46"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:38.056573 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "f9f50f86382941d7a810b00eb88fe3ac"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "cd1845135d60a667fb77fc0ffac427a9"
server_key_iv: "28449b91bf35795a97b3bec084252f46"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:38.058230 13325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:10:38.070897 13340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:38.071101 13325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:10:38.071178 13325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "f9f50f86382941d7a810b00eb88fe3ac"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "cd1845135d60a667fb77fc0ffac427a9"
server_key_iv: "28449b91bf35795a97b3bec084252f46"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:38.071246 13325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:38.082993 13325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:38.085004 13325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:38.085147 13325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:38.089882 13325 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:37469
I20260620 23:10:38.089989 13392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:37469 every 8 connection(s)
I20260620 23:10:38.090351 13325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:10:38.091147 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:38.093494 13393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Bootstrap starting.
I20260620 23:10:38.094105 13393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:38.094424 13393 log.cc:824] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:38.095184 13393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: No bootstrap required, opened a new log
I20260620 23:10:38.097172 13393 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:38.097330 13393 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:38.097401 13393 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f50f86382941d7a810b00eb88fe3ac, State: Initialized, Role: FOLLOWER
I20260620 23:10:38.097579 13393 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:38.097674 13393 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:10:38.097718 13393 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:10:38.097764 13393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:38.098278 13393 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:38.098399 13393 leader_election.cc:302] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9f50f86382941d7a810b00eb88fe3ac; no voters:
I20260620 23:10:38.098568 13393 leader_election.cc:288] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:10:38.098601 13396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:10:38.098696 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 13325
I20260620 23:10:38.098790 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:10:38.098855 13396 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 LEADER]: Becoming Leader. State: Replica: f9f50f86382941d7a810b00eb88fe3ac, State: Running, Role: LEADER
I20260620 23:10:38.098944 13393 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:38.099011 1153 external_mini_cluster.cc:1428] Setting key e7326f39774a8c4dd15dd625d0ee0d83
I20260620 23:10:38.098953 13396 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:38.099282 13396 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9f50f86382941d7a810b00eb88fe3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } } }
I20260620 23:10:38.099335 13396 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:38.099434 13396 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9f50f86382941d7a810b00eb88fe3ac. Latest consensus state: current_term: 1 leader_uuid: "f9f50f86382941d7a810b00eb88fe3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } } }
I20260620 23:10:38.099473 13396 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:38.100728 13393 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:10:38.104212 13400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:10:38.104964 13400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:10:38 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:38.111276 13400 catalog_manager.cc:1347] Generated new cluster ID: d8e2fc71102940e8bd9fab24ccdb73a7
I20260620 23:10:38.111320 13400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:10:38.113376 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:38.099602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.1.32.126:37469)
0620 23:10:38.099762 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:38.099771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:38.099943 (+ 172us) server_negotiation.cc:407] Connection header received
0620 23:10:38.100122 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:38.100128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:38.100191 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:38.100340 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:38.102490 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:38.104678 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:38.105958 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:38.106542 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:38.106620 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:38.109374 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:38.109388 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:38.109388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:38.109399 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:38.111561 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:38.111977 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:38.111980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:38.111980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:38.112092 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:38.112259 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:38.112259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:38.112259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:38.112518 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:38.112619 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:38.113060 (+ 441us) server_negotiation.cc:299] Negotiation successful
0620 23:10:38.113170 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260620 23:10:38.128225 13400 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:10:38.129019 13400 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:10:38.134387 13400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Generated new TSK 0
I20260620 23:10:38.134614 13400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:10:38.839870 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:37469
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:38.945335 13438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:38.945538 13438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:38.945571 13438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:38.946661 13438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:38.946722 13438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:38.946759 13438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:10:38.948107 13438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.32.84:53079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:37469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:38.948330 13438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:38.948524 13438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:38.952906 13446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:38.952885 13443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:38.953039 13444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:38.953737 13438 server_base.cc:1034] running on GCE node
I20260620 23:10:38.953943 13438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:38.954160 13438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:38.955314 13438 hybrid_clock.cc:648] HybridClock initialized: now 1781997038955294 us; error 39 us; skew 500 ppm
Jun 20 23:10:38 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:38.957842 13438 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:10:38.958338 13438 webserver.cc:460] Webserver started at http://127.1.32.65:46263/ using document root <none> and password file <none>
I20260620 23:10:38.958576 13438 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:38.958638 13438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:38.958765 13438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:10:38 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:38.970489 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "a332fd96e18845cc82dd885410905ea6"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "88473460a435027182766fca96d7b71a"
server_key_iv: "40bec917752a24b100a85ab6815115ed"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:38.971026 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "a332fd96e18845cc82dd885410905ea6"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "88473460a435027182766fca96d7b71a"
server_key_iv: "40bec917752a24b100a85ab6815115ed"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:38.976279 13438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260620 23:10:38.988274 13454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:38.988528 13438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:38.988673 13438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "a332fd96e18845cc82dd885410905ea6"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "88473460a435027182766fca96d7b71a"
server_key_iv: "40bec917752a24b100a85ab6815115ed"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:38.988827 13438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:39.002223 13438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:39.004632 13438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:39.004770 13438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:39.005031 13438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:39.005373 13438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:39.005466 13438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.005533 13438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:39.005580 13438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.012918 13438 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:35805
I20260620 23:10:39.013024 13568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:35805 every 8 connection(s)
I20260620 23:10:39.013388 13438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:10:39.015264 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 13438
I20260620 23:10:39.015350 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:10:39.015503 1153 external_mini_cluster.cc:1428] Setting key a26d1e4a8e1f285ba85c45e0bcfd9d30
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:39.025897 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.014065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:37469 (local address 127.1.32.65:37597)
0620 23:10:39.014281 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:39.014316 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:39.014879 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:39.015908 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:39.015912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:39.016139 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:39.016411 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.016418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.017367 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.017368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.018638 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.018640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.020134 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.020136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.020283 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.020801 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:39.020810 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:39.022412 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:39.024304 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.024311 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.024313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.024599 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.024821 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.024821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.024822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.024942 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.025226 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:39.025232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:39.025425 (+ 193us) client_negotiation.cc:769] Sending connection context
0620 23:10:39.025536 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 23:10:39.025679 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:10:39.026422 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.014092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:37597 (local address 127.1.32.126:37469)
0620 23:10:39.015086 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:39.015087 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:39.015095 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:10:39.015706 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:39.015707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:39.015723 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:39.015787 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:39.016506 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.017269 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.018727 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.019051 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.019108 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.022516 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:39.022517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:39.022517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:39.022525 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:39.024213 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.024687 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.024688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.024688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.024752 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.025015 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.025016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.025016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.025142 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:39.025200 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:39.026234 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0620 23:10:39.026315 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":82,"threads_started":1}
I20260620 23:10:39.027158 13569 heartbeater.cc:347] Connected to a master server at 127.1.32.126:37469
I20260620 23:10:39.027261 13569 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:39.027456 13569 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:39.028008 13357 ts_manager.cc:194] Registered new tserver with Master: a332fd96e18845cc82dd885410905ea6 (127.1.32.65:35805)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
I20260620 23:10:39.028883 13357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:37597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:10:39.045540 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:37469
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:39.149211 13575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:39.149443 13575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:39.149482 13575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:39.150842 13575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:39.150965 13575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:39.151031 13575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:10:39.152796 13575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.32.84:53079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:37469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:39.153043 13575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:39.153278 13575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:39.158437 13582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:39.158862 13575 server_base.cc:1034] running on GCE node
W20260620 23:10:39.158504 13583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:39.158645 13585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:39.159370 13575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:39.159639 13575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:39.162938 13575 hybrid_clock.cc:648] HybridClock initialized: now 1781997039162894 us; error 48 us; skew 500 ppm
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:39.166097 13575 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:10:39.166666 13575 webserver.cc:460] Webserver started at http://127.1.32.66:35225/ using document root <none> and password file <none>
I20260620 23:10:39.166985 13575 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:39.167079 13575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:39.167227 13575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:39.183382 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "2d0cd70de8324177b308e9571ed2ad89"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "84476e9551b048ec23bb5c677adbdb18"
server_key_iv: "d69a51dc64ff5ea73e510dad6afd2262"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.183806 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "2d0cd70de8324177b308e9571ed2ad89"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "84476e9551b048ec23bb5c677adbdb18"
server_key_iv: "d69a51dc64ff5ea73e510dad6afd2262"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.200917 13575 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.002s sys 0.000s
I20260620 23:10:39.227062 13592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.227277 13575 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.000s sys 0.000s
I20260620 23:10:39.227386 13575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "2d0cd70de8324177b308e9571ed2ad89"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "84476e9551b048ec23bb5c677adbdb18"
server_key_iv: "d69a51dc64ff5ea73e510dad6afd2262"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.227504 13575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:39.252348 13575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:39.254734 13575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:39.254923 13575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:39.255125 13575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:39.255441 13575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:39.255475 13575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.255515 13575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:39.255532 13575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.261694 13575 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:38417
I20260620 23:10:39.262120 13575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:10:39.264706 13706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:38417 every 8 connection(s)
I20260620 23:10:39.271481 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 13575
I20260620 23:10:39.271570 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:10:39.271764 1153 external_mini_cluster.cc:1428] Setting key ae6d44bf7b9a62c60991764d50f1f132
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:39.275557 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.265291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:41439 (local address 127.1.32.126:37469)
0620 23:10:39.265548 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:39.265550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:39.265585 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:10:39.266263 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:39.266265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:39.266278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:39.266306 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:39.267131 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.267945 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.269372 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.269662 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.269700 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.272275 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:39.272277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:39.272277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:39.272284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:39.273943 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.274422 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.274423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.274424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.274479 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.274732 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.274733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.274733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.274857 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:39.274938 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:39.275355 (+ 417us) server_negotiation.cc:299] Negotiation successful
0620 23:10:39.275460 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260620 23:10:39.276062 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.265298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:37469 (local address 127.1.32.66:41439)
0620 23:10:39.265463 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:39.265502 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:39.266143 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:39.266474 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:39.266478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:39.266715 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:39.267018 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.267025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.268064 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.268065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.269274 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.269277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.269790 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.269790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.269927 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.270517 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:39.270527 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:39.272171 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:39.274038 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.274044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.274046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.274331 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.274546 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.274546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.274546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.274664 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.274984 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:39.274989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:39.275201 (+ 212us) client_negotiation.cc:769] Sending connection context
0620 23:10:39.275678 (+ 477us) client_negotiation.cc:240] Negotiation successful
0620 23:10:39.275838 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":100,"threads_started":1}
I20260620 23:10:39.276453 13707 heartbeater.cc:347] Connected to a master server at 127.1.32.126:37469
I20260620 23:10:39.276543 13707 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:39.276713 13707 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:39.277030 13357 ts_manager.cc:194] Registered new tserver with Master: 2d0cd70de8324177b308e9571ed2ad89 (127.1.32.66:38417)
I20260620 23:10:39.277467 13357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:41439
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:10:39.309221 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:37469
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:39.428686 13712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:39.428996 13712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:39.429078 13712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:39.430804 13712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:39.430917 13712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:39.430997 13712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:10:39.436764 13712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.32.84:53079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:37469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:39.437247 13712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:39.437548 13712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:39.443742 13720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:39.443742 13717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:39.444869 13712 server_base.cc:1034] running on GCE node
W20260620 23:10:39.447325 13718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:39.447597 13712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:39.447834 13712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:39.448982 13712 hybrid_clock.cc:648] HybridClock initialized: now 1781997039448961 us; error 32 us; skew 500 ppm
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:39.451949 13712 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:10:39.452359 13712 webserver.cc:460] Webserver started at http://127.1.32.67:41155/ using document root <none> and password file <none>
I20260620 23:10:39.452571 13712 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:39.452615 13712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:39.452718 13712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:39.472370 13712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "1650e6a8bcdd4231ba310482e8f467a8"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "b83987225517e683fc47037f9bd62ad5"
server_key_iv: "7bb077b78aebf5f04e21724c20bf86ea"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.479246 13712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "1650e6a8bcdd4231ba310482e8f467a8"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "b83987225517e683fc47037f9bd62ad5"
server_key_iv: "7bb077b78aebf5f04e21724c20bf86ea"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.482091 13712 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 23:10:39.495537 13727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.496094 13712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:10:39.496198 13712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "1650e6a8bcdd4231ba310482e8f467a8"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "b83987225517e683fc47037f9bd62ad5"
server_key_iv: "7bb077b78aebf5f04e21724c20bf86ea"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.496290 13712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:39.531255 13712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:39.533874 13712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:39.534097 13712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:39.534399 13712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:39.534818 13712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:39.534943 13712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.535023 13712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:39.535080 13712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.541091 13712 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39203
I20260620 23:10:39.541540 13712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:10:39.541791 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 13712
I20260620 23:10:39.541868 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:10:39.542060 1153 external_mini_cluster.cc:1428] Setting key 9213ad087f3dcca9d66d2955b1fc00ff
I20260620 23:10:39.552000 13840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39203 every 8 connection(s)
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:39.562546 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.552238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:56501 (local address 127.1.32.126:37469)
0620 23:10:39.552281 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:39.552283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:39.552689 (+ 406us) server_negotiation.cc:407] Connection header received
0620 23:10:39.553922 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:39.553923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:39.553934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:39.553963 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:39.554743 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.555483 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.556787 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.557035 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.557071 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.559539 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:39.559540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:39.559540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:39.559547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:39.561128 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.561575 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.561576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.561576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.561627 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.561859 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.561860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.561860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.561994 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:39.562044 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:39.562373 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 23:10:39.562455 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:10:39.563231 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.552247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:37469 (local address 127.1.32.67:56501)
0620 23:10:39.552563 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:39.552605 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:39.553814 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:39.554137 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:39.554141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:39.554378 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:39.554642 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.554648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.555589 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.555590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.556697 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.556700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.557159 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.557160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.557278 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.557869 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:39.557879 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:39.559413 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:39.561227 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.561233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.561235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.561495 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.561691 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.561691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.561691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.561797 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.562085 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:39.562090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:39.562239 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 23:10:39.562850 (+ 611us) client_negotiation.cc:240] Negotiation successful
0620 23:10:39.563013 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":135,"threads_started":1}
I20260620 23:10:39.564270 13841 heartbeater.cc:347] Connected to a master server at 127.1.32.126:37469
I20260620 23:10:39.564350 13841 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:39.564524 13841 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:39.564888 13357 ts_manager.cc:194] Registered new tserver with Master: 1650e6a8bcdd4231ba310482e8f467a8 (127.1.32.67:39203)
I20260620 23:10:39.565325 13357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:56501
I20260620 23:10:39.574849 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:10:39.575667 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 13438
I20260620 23:10:39.582870 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 13575
I20260620 23:10:39.588282 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 13712
I20260620 23:10:39.592850 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 13325
I20260620 23:10:39.599679 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:37469
--webserver_interface=127.1.32.126
--webserver_port=42829
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:37469
--ranger_config_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:39.677958 13844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:39.678138 13844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:39.678158 13844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:39.679380 13844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:39.679419 13844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:39.679433 13844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:39.679445 13844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:39.679457 13844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:39.680912 13844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:37469
--ranger_config_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:37469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=42829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:39.681120 13844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:39.681337 13844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:39.685966 13850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:39.686007 13844 server_base.cc:1034] running on GCE node
W20260620 23:10:39.685974 13852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:39.686172 13849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:39.686389 13844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:39.686672 13844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:39.687829 13844 hybrid_clock.cc:648] HybridClock initialized: now 1781997039687817 us; error 30 us; skew 500 ppm
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:39.690253 13844 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:10:39.690634 13844 webserver.cc:460] Webserver started at http://127.1.32.126:42829/ using document root <none> and password file <none>
I20260620 23:10:39.690853 13844 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:39.690944 13844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:39.764483 13844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:39.765275 13860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.765537 13844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:39.765632 13844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "f9f50f86382941d7a810b00eb88fe3ac"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "cd1845135d60a667fb77fc0ffac427a9"
server_key_iv: "28449b91bf35795a97b3bec084252f46"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.766044 13844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:39.785770 13844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:39.787736 13844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:39.787837 13844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:39.791915 13844 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:37469
I20260620 23:10:39.791967 13912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:37469 every 8 connection(s)
I20260620 23:10:39.792304 13844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:10:39.793479 13913 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:10:39.794206 13913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Bootstrap starting.
I20260620 23:10:39.796068 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 13844
I20260620 23:10:39.796342 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:35805
--local_ip_for_outbound_sockets=127.1.32.65
--tserver_master_addrs=127.1.32.126:37469
--webserver_port=46263
--webserver_interface=127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:10:39.802048 13913 log.cc:824] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:39.803727 13913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:10:39.803994 13913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Bootstrap complete.
I20260620 23:10:39.806268 13913 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:39.806509 13913 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f50f86382941d7a810b00eb88fe3ac, State: Initialized, Role: FOLLOWER
I20260620 23:10:39.806702 13913 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:39.806792 13913 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:10:39.806842 13913 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:10:39.806905 13913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:10:39.807849 13913 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:39.807971 13913 leader_election.cc:302] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9f50f86382941d7a810b00eb88fe3ac; no voters:
I20260620 23:10:39.808145 13913 leader_election.cc:288] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:10:39.808203 13917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:10:39.808331 13917 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [term 2 LEADER]: Becoming Leader. State: Replica: f9f50f86382941d7a810b00eb88fe3ac, State: Running, Role: LEADER
I20260620 23:10:39.808404 13913 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:39.808434 13917 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } }
I20260620 23:10:39.808653 13918 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f9f50f86382941d7a810b00eb88fe3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } } }
I20260620 23:10:39.808655 13917 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9f50f86382941d7a810b00eb88fe3ac. Latest consensus state: current_term: 2 leader_uuid: "f9f50f86382941d7a810b00eb88fe3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f50f86382941d7a810b00eb88fe3ac" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 37469 } } }
I20260620 23:10:39.808845 13917 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:39.808833 13918 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:39.809194 13927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:10:39.809705 13927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:10:39.809811 13927 catalog_manager.cc:1259] Loaded cluster ID: d8e2fc71102940e8bd9fab24ccdb73a7
I20260620 23:10:39.809842 13927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:10:39.810132 13913 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:10:39.811046 13927 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:10:39.813216 13927 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Loaded TSK: 0
I20260620 23:10:39.813558 13927 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:10:39.896662 13915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:39.896849 13915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:39.896883 13915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:39.897933 13915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:39.897989 13915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:39.898027 13915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:10:39.899371 13915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:35805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=46263
--tserver_master_addrs=127.1.32.126:37469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:39.899601 13915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:39.899816 13915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:39.904836 13953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:39.905369 13951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:39.905725 13950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:39.905928 13915 server_base.cc:1034] running on GCE node
I20260620 23:10:39.906102 13915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:39.906383 13915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:39.907529 13915 hybrid_clock.cc:648] HybridClock initialized: now 1781997039907514 us; error 33 us; skew 500 ppm
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:39.915289 13915 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:10:39.915756 13915 webserver.cc:460] Webserver started at http://127.1.32.65:46263/ using document root <none> and password file <none>
I20260620 23:10:39.915975 13915 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:39.916038 13915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:39.935797 13915 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.001s
I20260620 23:10:39.937011 13961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.937162 13915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:10:39.937229 13915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "a332fd96e18845cc82dd885410905ea6"
format_stamp: "Formatted at 2026-06-20 23:10:38 on dist-test-slave-fzjm"
server_key: "88473460a435027182766fca96d7b71a"
server_key_iv: "40bec917752a24b100a85ab6815115ed"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:39.937630 13915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:39.959622 13915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:39.961846 13915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:39.961997 13915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:39.962198 13915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:39.962497 13915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:39.962535 13915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.962563 13915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:39.962577 13915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:39.969498 13915 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:35805
I20260620 23:10:39.969668 14074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:35805 every 8 connection(s)
I20260620 23:10:39.969940 13915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:10:39.971721 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 13915
I20260620 23:10:39.971997 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:38417
--local_ip_for_outbound_sockets=127.1.32.66
--tserver_master_addrs=127.1.32.126:37469
--webserver_port=35225
--webserver_interface=127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:10:39 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:39.983176 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.970680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:37469 (local address 127.1.32.65:37179)
0620 23:10:39.970863 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:39.970928 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:39.971856 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:39.972517 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:39.972521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:39.972737 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:39.972901 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.972905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.974415 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.974417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.975753 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:39.975757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.976677 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.976679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:39.976822 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.977365 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:39.977376 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:39.979072 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:39.981317 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.981324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.981326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.981650 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.981946 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:39.981948 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:39.981948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:39.982087 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:39.982552 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:39.982557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:39.982688 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 23:10:39.982771 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 23:10:39.982942 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260620 23:10:39.983812 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:39.971048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:37179 (local address 127.1.32.126:37469)
0620 23:10:39.971288 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:39.971296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:39.971339 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:10:39.972170 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:39.972178 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:39.972237 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:39.972365 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:39.973007 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.974290 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.975851 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:39.976517 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:39.978090 (+ 1573us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:39.979191 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:39.979196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:39.979196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:39.979208 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:39.981224 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.981760 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.981763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.981764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.981867 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:39.982181 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:39.982182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:39.982183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:39.982465 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:39.983418 (+ 953us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:39.983491 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:10:39.983617 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
I20260620 23:10:39.984377 14075 heartbeater.cc:347] Connected to a master server at 127.1.32.126:37469
I20260620 23:10:39.984465 14075 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:39.984637 14075 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
W20260620 23:10:39.985436 14075 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:37469 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:10:39.985508 14075 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:39.985572 14075 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:39.985870 14075 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:39.985955 14075 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
W20260620 23:10:39.986222 14075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:10:40.066182 14079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:40.066425 14079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:40.066457 14079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:40.068001 14079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:40.068040 14079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:40.068085 14079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:10:40.070055 14079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:38417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=35225
--tserver_master_addrs=127.1.32.126:37469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:40.070273 14079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:40.070519 14079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:40.075620 14084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:40.076004 14079 server_base.cc:1034] running on GCE node
W20260620 23:10:40.079571 14087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:40.079749 14085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:40.079981 14079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:40.080235 14079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:40.081395 14079 hybrid_clock.cc:648] HybridClock initialized: now 1781997040081363 us; error 56 us; skew 500 ppm
Jun 20 23:10:40 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:40.084465 14079 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:10:40.084929 14079 webserver.cc:460] Webserver started at http://127.1.32.66:35225/ using document root <none> and password file <none>
I20260620 23:10:40.085183 14079 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:40.085259 14079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:10:40 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:40.115091 14079 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.001s sys 0.000s
I20260620 23:10:40.116124 14095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:40.116272 14079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:10:40.116348 14079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "2d0cd70de8324177b308e9571ed2ad89"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "84476e9551b048ec23bb5c677adbdb18"
server_key_iv: "d69a51dc64ff5ea73e510dad6afd2262"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:40.116770 14079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:40.145828 14079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:40.148121 14079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:40.148275 14079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:40.148507 14079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:40.148865 14079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:40.148916 14079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:40.148954 14079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:40.148980 14079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:40.156101 14079 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:38417
I20260620 23:10:40.156203 14208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:38417 every 8 connection(s)
I20260620 23:10:40.156515 14079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:10:40.158057 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14079
I20260620 23:10:40.158331 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.32.84:53079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:39203
--local_ip_for_outbound_sockets=127.1.32.67
--tserver_master_addrs=127.1.32.126:37469
--webserver_port=41155
--webserver_interface=127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:10:40 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:40.167492 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:40.157281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:59059 (local address 127.1.32.126:37469)
0620 23:10:40.158281 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:40.158284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:40.158298 (+ 14us) server_negotiation.cc:407] Connection header received
0620 23:10:40.158377 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:40.158379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:40.158394 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:40.158429 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:40.159308 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.160007 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.161107 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.161417 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.161496 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:40.164133 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:40.164135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:40.164135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:40.164142 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:40.165850 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:40.166314 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:40.166315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:40.166316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:40.166385 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:40.166707 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:40.166708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:40.166708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:40.166837 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:40.166886 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:40.167295 (+ 409us) server_negotiation.cc:299] Negotiation successful
0620 23:10:40.167395 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":968}
I20260620 23:10:40.168236 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:40.157297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:37469 (local address 127.1.32.66:59059)
0620 23:10:40.157486 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:40.157519 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:40.158255 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:40.158630 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:40.158634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:40.158866 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:40.159201 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:40.159207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.160099 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.160100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:40.161031 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:40.161034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.161601 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.161602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:40.161750 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:40.162385 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:40.162394 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:40.164024 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:40.165940 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:40.165946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:40.165948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:40.166225 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:40.166476 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:40.166477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:40.166477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:40.166606 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:40.166967 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:40.166973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:40.167129 (+ 156us) client_negotiation.cc:769] Sending connection context
0620 23:10:40.167839 (+ 710us) client_negotiation.cc:240] Negotiation successful
0620 23:10:40.168003 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260620 23:10:40.168924 14209 heartbeater.cc:347] Connected to a master server at 127.1.32.126:37469
I20260620 23:10:40.169009 14209 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:40.169178 14209 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
W20260620 23:10:40.169842 14209 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:37469 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:10:40.169905 14209 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:40.169960 14209 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:40.170291 14209 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:40.170375 14209 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
W20260620 23:10:40.170681 14209 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:10:40.252990 14212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:40.253183 14212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:40.253204 14212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:40.254416 14212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:40.254454 14212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:40.254487 14212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:10:40.256027 14212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.32.84:53079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:39203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=41155
--tserver_master_addrs=127.1.32.126:37469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:40.256261 14212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:40.256520 14212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:40.261372 14217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:40.261382 14220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:40.263126 14218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:40.265381 14212 server_base.cc:1034] running on GCE node
I20260620 23:10:40.265597 14212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:40.265836 14212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:40.266947 14212 hybrid_clock.cc:648] HybridClock initialized: now 1781997040266931 us; error 24 us; skew 500 ppm
Jun 20 23:10:40 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:40.269476 14212 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:10:40.269999 14212 webserver.cc:460] Webserver started at http://127.1.32.67:41155/ using document root <none> and password file <none>
I20260620 23:10:40.270272 14212 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:40.270347 14212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:10:40 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:40.291190 14212 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260620 23:10:40.292456 14229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:40.292666 14212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:40.292804 14212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "1650e6a8bcdd4231ba310482e8f467a8"
format_stamp: "Formatted at 2026-06-20 23:10:39 on dist-test-slave-fzjm"
server_key: "b83987225517e683fc47037f9bd62ad5"
server_key_iv: "7bb077b78aebf5f04e21724c20bf86ea"
server_key_version: "kuduclusterkey@0"
I20260620 23:10:40.293341 14212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:40.307379 14212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:40.309553 14212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:40.309720 14212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:40.309954 14212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:40.310276 14212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:40.310319 14212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:40.310352 14212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:40.310385 14212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:40.316967 14212 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39203
I20260620 23:10:40.317073 14342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39203 every 8 connection(s)
I20260620 23:10:40.317399 14212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:10:40 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:40.327311 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14212
I20260620 23:10:40.327517 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:40.318138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:33969 (local address 127.1.32.126:37469)
0620 23:10:40.318186 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:40.318188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:40.318453 (+ 265us) server_negotiation.cc:407] Connection header received
0620 23:10:40.319129 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:40.319131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:40.319145 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:40.319173 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:40.320058 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.320814 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.322217 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.322379 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.322425 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:40.324547 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:40.324548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:40.324548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:40.324555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:40.326044 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:40.326474 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:40.326475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:40.326475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:40.326543 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:40.326793 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:40.326794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:40.326794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:40.326938 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:40.327004 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:40.327343 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 23:10:40.327423 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:10:40.328058 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:40.318148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:37469 (local address 127.1.32.67:33969)
0620 23:10:40.318306 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:40.318350 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:40.318988 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:40.319345 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:40.319349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:40.319663 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:40.319967 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:40.319973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.320915 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.320916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:40.322122 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:40.322125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.322475 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.322476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:40.322611 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:40.323181 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:40.323189 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:40.324450 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:40.326125 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:40.326145 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:40.326146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:40.326332 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:40.326615 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:40.326615 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:40.326616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:40.326723 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:40.327033 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:40.327039 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:40.327187 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 23:10:40.327305 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 23:10:40.327447 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260620 23:10:40.328498 14343 heartbeater.cc:347] Connected to a master server at 127.1.32.126:37469
I20260620 23:10:40.328614 14343 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:40.328830 14343 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
W20260620 23:10:40.329484 14343 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:37469 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:10:40.329547 14343 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:40.329599 14343 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:40.329934 14343 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:40.330003 14343 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
W20260620 23:10:40.330231 14343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:10:40.334738 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:40.327880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51600 (local address 127.1.32.126:37469)
0620 23:10:40.327913 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:40.327915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:40.328014 (+ 99us) server_negotiation.cc:407] Connection header received
0620 23:10:40.328082 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:40.328083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:40.328091 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:40.328134 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:40.329116 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.329618 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.330508 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:40.330681 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:40.330740 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:40.331938 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:40.331939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:40.331939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:40.331945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:40.333281 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:40.333986 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:40.333988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:40.333988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:40.334040 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:40.334248 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:40.334249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:40.334250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:40.334373 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:40.334537 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:40.334572 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 23:10:40.334642 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Jun 20 23:10:40 dist-test-slave-fzjm krb5kdc[12881](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:40.986423 14075 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:40.986572 14075 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:40.986871 13877 ts_manager.cc:194] Registered new tserver with Master: a332fd96e18845cc82dd885410905ea6 (127.1.32.65:35805)
I20260620 23:10:40.987875 13877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:37179
I20260620 23:10:41.170836 14209 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:41.171037 14209 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:41.171295 13877 ts_manager.cc:194] Registered new tserver with Master: 2d0cd70de8324177b308e9571ed2ad89 (127.1.32.66:38417)
I20260620 23:10:41.171808 13877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:59059
Jun 20 23:10:41 dist-test-slave-fzjm krb5kdc[12881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997040, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for HTTP/127.1.32.84@KRBTEST.COM
I20260620 23:10:41.330415 14343 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:41.330585 14343 heartbeater.cc:511] Master 127.1.32.126:37469 requested a full tablet report, sending...
I20260620 23:10:41.330987 13877 ts_manager.cc:194] Registered new tserver with Master: 1650e6a8bcdd4231ba310482e8f467a8 (127.1.32.67:39203)
I20260620 23:10:41.331518 13877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:33969
I20260620 23:10:41.333772 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:10:41.342504 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.334733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.1.32.126:37469)
0620 23:10:41.334842 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:41.334844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:41.334864 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:10:41.334963 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:41.334964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:41.334975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:41.335047 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:41.335752 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.336503 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.337505 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.337822 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.337902 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.339460 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:41.339462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:41.339462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:41.339469 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:41.341258 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:41.341648 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:41.341649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:41.341650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:41.341708 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:41.341935 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:41.341935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:41.341936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:41.342073 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:41.342171 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:41.342313 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 23:10:41.342401 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":60,"threads_started":1}
I20260620 23:10:41.344847 13877 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:10:41.347185 13877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:10:41.354907 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.350105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:35805 (local address 127.0.0.1:55228)
0620 23:10:41.350727 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:41.350764 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:41.350822 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:41.351106 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:41.351109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:41.351111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:41.351173 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.351175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.352383 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.352384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.353614 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.353616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.354571 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.354573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.354654 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.354669 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 23:10:41.354744 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 23:10:41.354792 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":94,"threads_started":1}
I20260620 23:10:41.355212 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.350354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.1.32.65:35805)
0620 23:10:41.350672 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:41.350677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:41.350919 (+ 242us) server_negotiation.cc:407] Connection header received
0620 23:10:41.350954 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:41.350956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:41.350972 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:41.351033 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:41.351251 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.352295 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.353710 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.354484 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.354560 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.354618 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:41.354772 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 23:10:41.354822 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":46,"threads_started":1}
I20260620 23:10:41.355952 14009 tablet_service.cc:1467] Processing CreateTablet for tablet be1d254a302640b7871f93b1a16c738b (DEFAULT_TABLE table=test-table [id=7e87e32879de4980b2ab49dd71f5f7c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:41.356307 14009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be1d254a302640b7871f93b1a16c738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:41.360219 14375 tablet_bootstrap.cc:492] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6: Bootstrap starting.
I20260620 23:10:41.360828 14375 tablet_bootstrap.cc:654] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:41.361120 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.350533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39203 (local address 127.0.0.1:41070)
0620 23:10:41.351354 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:41.351360 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:41.351387 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:41.357631 (+ 6244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:41.357632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:41.357633 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:41.357669 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.357670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.358839 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.358840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.359886 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.359888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.360878 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.360879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.360974 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.360978 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:10:41.361022 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:10:41.361054 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":72,"threads_started":1}
I20260620 23:10:41.361140 14375 log.cc:824] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:41.361516 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.350668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.1.32.67:39203)
0620 23:10:41.357405 (+ 6737us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:41.357410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:41.357421 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:10:41.357454 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:41.357455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:41.357470 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:41.357544 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:41.357736 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.358754 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.359977 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.360731 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.360790 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.360836 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:41.361409 (+ 573us) server_negotiation.cc:299] Negotiation successful
0620 23:10:41.361447 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6702,"thread_start_us":55,"threads_started":1}
I20260620 23:10:41.362083 14277 tablet_service.cc:1467] Processing CreateTablet for tablet be1d254a302640b7871f93b1a16c738b (DEFAULT_TABLE table=test-table [id=7e87e32879de4980b2ab49dd71f5f7c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:41.362421 14277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be1d254a302640b7871f93b1a16c738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:41.362505 14375 tablet_bootstrap.cc:492] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6: No bootstrap required, opened a new log
I20260620 23:10:41.362555 14375 ts_tablet_manager.cc:1397] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:10:41.365324 14375 raft_consensus.cc:348] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.365482 14375 raft_consensus.cc:374] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:41.365522 14375 raft_consensus.cc:729] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a332fd96e18845cc82dd885410905ea6, State: Initialized, Role: FOLLOWER
I20260620 23:10:41.365588 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.350108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:38417 (local address 127.0.0.1:50590)
0620 23:10:41.351261 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:41.351269 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:41.351298 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:41.361951 (+ 10653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:41.361952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:41.361955 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:41.361990 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.361991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.363293 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.363294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.364339 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.364342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.365388 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.365389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.365451 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.365455 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:10:41.365487 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 23:10:41.365524 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"thread_start_us":86,"threads_started":1}
I20260620 23:10:41.365677 14375 consensus_queue.cc:260] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.365958 14375 ts_tablet_manager.cc:1428] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:10:41.365969 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.350367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.1.32.66:38417)
0620 23:10:41.361784 (+ 11417us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:41.361787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:41.361797 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:10:41.361829 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:41.361830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:41.361845 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:41.361931 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:41.362080 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.363066 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.364462 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.365272 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.365758 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.365821 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:41.365848 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 23:10:41.365892 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11381,"thread_start_us":63,"threads_started":1}
I20260620 23:10:41.366082 14075 heartbeater.cc:503] Master 127.1.32.126:37469 was elected leader, sending a full tablet report...
I20260620 23:10:41.366566 14143 tablet_service.cc:1467] Processing CreateTablet for tablet be1d254a302640b7871f93b1a16c738b (DEFAULT_TABLE table=test-table [id=7e87e32879de4980b2ab49dd71f5f7c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:41.366847 14143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be1d254a302640b7871f93b1a16c738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:41.366933 14377 tablet_bootstrap.cc:492] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Bootstrap starting.
I20260620 23:10:41.367794 14377 tablet_bootstrap.cc:654] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:41.368095 14377 log.cc:824] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:41.369249 14377 tablet_bootstrap.cc:492] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: No bootstrap required, opened a new log
I20260620 23:10:41.369315 14377 ts_tablet_manager.cc:1397] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:10:41.371098 14380 tablet_bootstrap.cc:492] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89: Bootstrap starting.
I20260620 23:10:41.371723 14380 tablet_bootstrap.cc:654] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:41.371706 14377 raft_consensus.cc:348] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.371868 14377 raft_consensus.cc:374] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:41.371901 14377 raft_consensus.cc:729] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1650e6a8bcdd4231ba310482e8f467a8, State: Initialized, Role: FOLLOWER
I20260620 23:10:41.372087 14377 consensus_queue.cc:260] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.372108 14380 log.cc:824] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:41.372344 14377 ts_tablet_manager.cc:1428] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:10:41.372798 14343 heartbeater.cc:503] Master 127.1.32.126:37469 was elected leader, sending a full tablet report...
I20260620 23:10:41.372851 14380 tablet_bootstrap.cc:492] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89: No bootstrap required, opened a new log
I20260620 23:10:41.372906 14380 ts_tablet_manager.cc:1397] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:10:41.375303 14380 raft_consensus.cc:348] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.375461 14380 raft_consensus.cc:374] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:41.375507 14380 raft_consensus.cc:729] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0cd70de8324177b308e9571ed2ad89, State: Initialized, Role: FOLLOWER
I20260620 23:10:41.375626 14380 consensus_queue.cc:260] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.375859 14380 ts_tablet_manager.cc:1428] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:10:41.375919 14209 heartbeater.cc:503] Master 127.1.32.126:37469 was elected leader, sending a full tablet report...
I20260620 23:10:41.385250 14381 raft_consensus.cc:482] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:41.385334 14381 raft_consensus.cc:504] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.385625 14381 leader_election.cc:288] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d0cd70de8324177b308e9571ed2ad89 (127.1.32.66:38417), a332fd96e18845cc82dd885410905ea6 (127.1.32.65:35805)
I20260620 23:10:41.389667 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.385987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:44657 (local address 127.1.32.66:38417)
0620 23:10:41.386066 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:41.386068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:41.386079 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:10:41.386174 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:41.386175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:41.386187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:41.386213 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:41.386406 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.387144 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.388769 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.389312 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.389352 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.389361 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:41.389573 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 23:10:41.389606 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:10:41.389706 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.385966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:55991 (local address 127.1.32.65:35805)
0620 23:10:41.386036 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:41.386038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:41.386250 (+ 212us) server_negotiation.cc:407] Connection header received
0620 23:10:41.386342 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:41.386343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:41.386354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:41.386422 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:41.386611 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.387306 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.388767 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.389285 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.389332 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.389343 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:41.389605 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 23:10:41.389644 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:10:41.389966 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.385785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:38417 (local address 127.1.32.67:44657)
0620 23:10:41.386035 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:41.386046 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:41.386093 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:41.386272 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:41.386273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:41.386274 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:41.386342 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.386343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.387466 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.387467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.388673 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.388675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.389433 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.389434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.389500 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.389503 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:10:41.389789 (+ 286us) client_negotiation.cc:240] Negotiation successful
0620 23:10:41.389872 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":12,"thread_start_us":103,"threads_started":1}
I20260620 23:10:41.390252 14163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1d254a302640b7871f93b1a16c738b" candidate_uuid: "1650e6a8bcdd4231ba310482e8f467a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d0cd70de8324177b308e9571ed2ad89" is_pre_election: true
I20260620 23:10:41.390388 14163 raft_consensus.cc:2454] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1650e6a8bcdd4231ba310482e8f467a8 in term 0.
I20260620 23:10:41.390611 14232 leader_election.cc:302] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1650e6a8bcdd4231ba310482e8f467a8, 2d0cd70de8324177b308e9571ed2ad89; no voters:
I20260620 23:10:41.390724 14381 raft_consensus.cc:2781] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:10:41.390765 14381 raft_consensus.cc:482] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:10:41.390785 14381 raft_consensus.cc:3037] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:41.391669 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.385789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:35805 (local address 127.1.32.67:55991)
0620 23:10:41.386190 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:41.386200 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:41.386274 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:41.386482 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:41.386483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:41.386484 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:41.386545 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.386547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.387415 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.387416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.388666 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:41.388669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.389442 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.389443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:41.389518 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.389521 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:10:41.391550 (+ 2029us) client_negotiation.cc:240] Negotiation successful
0620 23:10:41.391592 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":44,"thread_start_us":94,"threads_started":1}
I20260620 23:10:41.391716 14381 raft_consensus.cc:504] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.391836 14381 leader_election.cc:288] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [CANDIDATE]: Term 1 election: Requested vote from peers 2d0cd70de8324177b308e9571ed2ad89 (127.1.32.66:38417), a332fd96e18845cc82dd885410905ea6 (127.1.32.65:35805)
I20260620 23:10:41.392130 14029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1d254a302640b7871f93b1a16c738b" candidate_uuid: "1650e6a8bcdd4231ba310482e8f467a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a332fd96e18845cc82dd885410905ea6" is_pre_election: true
I20260620 23:10:41.392290 14029 raft_consensus.cc:2454] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1650e6a8bcdd4231ba310482e8f467a8 in term 0.
I20260620 23:10:41.392373 14028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1d254a302640b7871f93b1a16c738b" candidate_uuid: "1650e6a8bcdd4231ba310482e8f467a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a332fd96e18845cc82dd885410905ea6"
I20260620 23:10:41.392457 14028 raft_consensus.cc:3037] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:41.392637 14163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be1d254a302640b7871f93b1a16c738b" candidate_uuid: "1650e6a8bcdd4231ba310482e8f467a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d0cd70de8324177b308e9571ed2ad89"
I20260620 23:10:41.392701 14163 raft_consensus.cc:3037] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:41.393255 14028 raft_consensus.cc:2454] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1650e6a8bcdd4231ba310482e8f467a8 in term 1.
I20260620 23:10:41.393510 14163 raft_consensus.cc:2454] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1650e6a8bcdd4231ba310482e8f467a8 in term 1.
I20260620 23:10:41.393507 14230 leader_election.cc:302] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1650e6a8bcdd4231ba310482e8f467a8, a332fd96e18845cc82dd885410905ea6; no voters:
I20260620 23:10:41.393680 14381 raft_consensus.cc:2781] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:10:41.393821 14381 raft_consensus.cc:686] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 1 LEADER]: Becoming Leader. State: Replica: 1650e6a8bcdd4231ba310482e8f467a8, State: Running, Role: LEADER
I20260620 23:10:41.393900 14381 consensus_queue.cc:237] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } }
I20260620 23:10:41.394531 13877 catalog_manager.cc:5495] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1650e6a8bcdd4231ba310482e8f467a8 (127.1.32.67). New cstate: current_term: 1 leader_uuid: "1650e6a8bcdd4231ba310482e8f467a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1650e6a8bcdd4231ba310482e8f467a8" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260620 23:10:41.401784 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.398918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.1.32.67:39203)
0620 23:10:41.398954 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:41.398956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:41.398965 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:10:41.399022 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:41.399023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:41.399040 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:41.399112 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:10:41.399272 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.399926 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.400944 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:41.401227 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:41.401297 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:41.401479 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:10:41.401593 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:10:41.401670 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:41.401709 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 23:10:41.401726 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:10:41.403353 14028 raft_consensus.cc:1261] T be1d254a302640b7871f93b1a16c738b P a332fd96e18845cc82dd885410905ea6 [term 1 FOLLOWER]: Refusing update from remote peer 1650e6a8bcdd4231ba310482e8f467a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:41.403412 14163 raft_consensus.cc:1261] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 1 FOLLOWER]: Refusing update from remote peer 1650e6a8bcdd4231ba310482e8f467a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:41.403731 14386 consensus_queue.cc:1035] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d0cd70de8324177b308e9571ed2ad89" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 38417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:41.403895 14386 consensus_queue.cc:1035] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a332fd96e18845cc82dd885410905ea6" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:41.406987 14389 mvcc.cc:205] Tried to move back new op lower bound from 7299059881585864704 to 7299059881549930496. Current Snapshot: MvccSnapshot[applied={T|T < 7299059881585864704}]
I20260620 23:10:41.407714 14391 mvcc.cc:205] Tried to move back new op lower bound from 7299059881585864704 to 7299059881549930496. Current Snapshot: MvccSnapshot[applied={T|T < 7299059881585864704}]
I20260620 23:10:41.408931 14390 mvcc.cc:205] Tried to move back new op lower bound from 7299059881585864704 to 7299059881549930496. Current Snapshot: MvccSnapshot[applied={T|T < 7299059881585864704}]
I20260620 23:10:41.409726 13876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:10:41.409808 13876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:10:41.410487 13876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9f50f86382941d7a810b00eb88fe3ac: Sending DeleteTablet for 3 replicas of tablet be1d254a302640b7871f93b1a16c738b
I20260620 23:10:41.411022 14143 tablet_service.cc:1514] Processing DeleteTablet for tablet be1d254a302640b7871f93b1a16c738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:41 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:50590
I20260620 23:10:41.411227 14277 tablet_service.cc:1514] Processing DeleteTablet for tablet be1d254a302640b7871f93b1a16c738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:41 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:41070
I20260620 23:10:41.411430 14009 tablet_service.cc:1514] Processing DeleteTablet for tablet be1d254a302640b7871f93b1a16c738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:41 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:55228
I20260620 23:10:41.411890 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 13915
I20260620 23:10:41.415401 14398 tablet_replica.cc:330] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89: stopping tablet replica
I20260620 23:10:41.415524 14398 raft_consensus.cc:2229] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:10:41.415620 14398 raft_consensus.cc:2258] T be1d254a302640b7871f93b1a16c738b P 2d0cd70de8324177b308e9571ed2ad89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:41.415766 14399 tablet_replica.cc:330] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: stopping tablet replica
I20260620 23:10:41.415858 14399 raft_consensus.cc:2229] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:10:41.415969 14399 raft_consensus.cc:2258] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:41.420236 14399 ts_tablet_manager.cc:1905] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:10:41.422114 14399 ts_tablet_manager.cc:1918] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260620 23:10:41.422111 13861 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:35805 (error 108)
I20260620 23:10:41.422194 14399 log.cc:1196] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/be1d254a302640b7871f93b1a16c738b
I20260620 23:10:41.422533 14399 ts_tablet_manager.cc:1939] T be1d254a302640b7871f93b1a16c738b P 1650e6a8bcdd4231ba310482e8f467a8: Deleting consensus metadata
I20260620 23:10:41.422941 13864 catalog_manager.cc:4841] TS 1650e6a8bcdd4231ba310482e8f467a8 (127.1.32.67:39203): tablet be1d254a302640b7871f93b1a16c738b (table test-table [id=7e87e32879de4980b2ab49dd71f5f7c6]) successfully deleted
I20260620 23:10:41.423146 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14079
I20260620 23:10:41.431041 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14212
I20260620 23:10:41.432428 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.432285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:35805 (local address 127.0.0.1:55236)
0620 23:10:41.432337 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:41.432377 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:10:41.432554 13861 catalog_manager.cc:4584] TS a332fd96e18845cc82dd885410905ea6 (127.1.32.65:35805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be1d254a302640b7871f93b1a16c738b: Network error: Client connection negotiation failed: client connection to 127.1.32.65:35805: connect: Connection refused (error 111)
I20260620 23:10:41.432835 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:41.432757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:38417 (local address 127.0.0.1:50606)
0620 23:10:41.432794 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:41.432814 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:38417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 23:10:41.437003 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 13844
I20260620 23:10:41.443758 1153 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 23:10:41.787681 1153 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 23:10:41.787770 1153 mini_ranger.cc:66] Stopping Ranger...
I20260620 23:10:42.131726 1153 mini_ranger.cc:68] Stopped Ranger
2026-06-20 23:10:42.132 UTC [12911] LOG: received smart shutdown request
2026-06-20 23:10:42.135 UTC [12911] LOG: background worker "logical replication launcher" (PID 12920) exited with exit code 1
2026-06-20 23:10:42.135 UTC [12915] LOG: shutting down
2026-06-20 23:10:42.279 UTC [12911] LOG: database system is shut down
2026-06-20T23:10:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46043 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:10:42 dist-test-slave-fzjm krb5kdc[14410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:10:42 dist-test-slave-fzjm krb5kdc[14410](info): set up 2 sockets
Jun 20 23:10:42 dist-test-slave-fzjm krb5kdc[14410](info): commencing operation
krb5kdc: starting...
W20260620 23:10:43.402491 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:10:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:10:43.509212 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38631
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:38631,127.1.32.125:41309,127.1.32.124:43203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:43.585614 14423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:43.585807 14423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:43.585826 14423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:43.586961 14423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:43.586999 14423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:43.587013 14423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:43.587025 14423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:43.587072 14423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:43.588503 14423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:38631,127.1.32.125:41309,127.1.32.124:43203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:43.588766 14423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:43.588997 14423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:43.593743 14428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:43.593833 14423 server_base.cc:1034] running on GCE node
W20260620 23:10:43.593727 14429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:43.593716 14431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:43.594409 14423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:43.594692 14423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:43.595856 14423 hybrid_clock.cc:648] HybridClock initialized: now 1781997043595858 us; error 52 us; skew 500 ppm
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:43.598348 14423 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:10:43.598783 14423 webserver.cc:460] Webserver started at http://127.1.32.126:45033/ using document root <none> and password file <none>
I20260620 23:10:43.599079 14423 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:43.599157 14423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:43.599287 14423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:43.600279 14423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "67f65aff26f247c6ba6f61abaac6d7ec"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "ee3cb73d1acaccbc034a0f4e9e7a55bb"
server_key_iv: "467cfaf7aa30691248cb7511b582a2c2"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.600621 14423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "67f65aff26f247c6ba6f61abaac6d7ec"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "ee3cb73d1acaccbc034a0f4e9e7a55bb"
server_key_iv: "467cfaf7aa30691248cb7511b582a2c2"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.601856 14423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:10:43.602502 14438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:43.602728 14423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:10:43.602821 14423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "67f65aff26f247c6ba6f61abaac6d7ec"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "ee3cb73d1acaccbc034a0f4e9e7a55bb"
server_key_iv: "467cfaf7aa30691248cb7511b582a2c2"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.602934 14423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:43.621977 14423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:43.624145 14423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:43.624295 14423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:43.628206 14423 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:38631
I20260620 23:10:43.628239 14490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:38631 every 8 connection(s)
I20260620 23:10:43.628623 14423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:10:43.629417 14491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:43.630832 14491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:10:43.633409 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14423
I20260620 23:10:43.633487 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:10:43.633685 1153 external_mini_cluster.cc:1428] Setting key c4169d1730e0e69629602564b4507f91
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:43.640077 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.631437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38631 (local address 127.0.0.1:52184)
0620 23:10:43.631557 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.631590 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.632006 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.632216 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.632217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.632442 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.632635 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.632638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.633633 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.633634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.634343 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.634345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.634876 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.634877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.634966 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.635458 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.635466 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.636956 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.638648 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.638654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.638656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.638847 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.639106 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.639107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.639107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.639189 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.639512 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.639516 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.639571 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.639668 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.639905 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":118,"thread_start_us":56,"threads_started":1}
I20260620 23:10:43.640085 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.631570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.1.32.126:38631)
0620 23:10:43.631684 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.631688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.631718 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:10:43.632122 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.632124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.632139 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.632211 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.632713 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.633556 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.634448 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.634778 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.634930 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.637049 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.637055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.637056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.637062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.638532 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.638961 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.638963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.638964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.639040 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.639251 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.639252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.639253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.639445 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.639529 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.639712 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.639852 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":85,"thread_start_us":61,"threads_started":1}
WARNING: no policy specified for kudu/127.1.32.125@KRBTEST.COM; defaulting to no policy
I20260620 23:10:43.640810 14491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:10:43.641121 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.641036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38220)
0620 23:10:43.641065 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.641103 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260620 23:10:43.641260 14441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
I20260620 23:10:43.641734 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.641643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38236)
0620 23:10:43.641691 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.641717 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 23:10:43.641872 14491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:41309: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab.
Entry for principal kudu/127.1.32.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab.
I20260620 23:10:43.655900 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:41309
--webserver_interface=127.1.32.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab
--principal=kudu/127.1.32.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:38631,127.1.32.125:41309,127.1.32.124:43203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:10:43.691043 14491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } attempt: 1
I20260620 23:10:43.691519 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.691374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38244)
0620 23:10:43.691449 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.691489 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 23:10:43.691901 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.691795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38246)
0620 23:10:43.691853 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.691882 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:10:43.692005 14491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:41309: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
W20260620 23:10:43.733109 14498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:43.733340 14498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:43.733381 14498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:43.734449 14498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:43.734513 14498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:43.734536 14498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:43.734562 14498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:43.734591 14498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:43.735996 14498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:38631,127.1.32.125:41309,127.1.32.124:43203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab
--principal=kudu/127.1.32.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:41309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.32.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:43.736248 14498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:43.736461 14498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:43.741034 14506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:43.741072 14503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:43.741096 14498 server_base.cc:1034] running on GCE node
W20260620 23:10:43.741043 14504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:43.741477 14498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:43.741796 14498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:43.742973 14498 hybrid_clock.cc:648] HybridClock initialized: now 1781997043742955 us; error 28 us; skew 500 ppm
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:43.745416 14498 init.cc:377] Logged in from keytab as kudu/127.1.32.125@KRBTEST.COM (short username kudu)
I20260620 23:10:43.745849 14498 webserver.cc:460] Webserver started at http://127.1.32.125:37377/ using document root <none> and password file <none>
I20260620 23:10:43.746069 14498 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:43.746129 14498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:43.746273 14498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:43.747254 14498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data/instance:
uuid: "04386a7525a349b5850f303fce29236d"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "4a9ba3b250c5843a125913054f3c00c6"
server_key_iv: "24aac64a17ea4ea330d91d91b8dd56af"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.747597 14498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/wal/instance:
uuid: "04386a7525a349b5850f303fce29236d"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "4a9ba3b250c5843a125913054f3c00c6"
server_key_iv: "24aac64a17ea4ea330d91d91b8dd56af"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.748821 14498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:10:43.749527 14513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:43.749707 14498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:10:43.749800 14498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/wal
uuid: "04386a7525a349b5850f303fce29236d"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "4a9ba3b250c5843a125913054f3c00c6"
server_key_iv: "24aac64a17ea4ea330d91d91b8dd56af"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.749874 14498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:43.760156 14491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } attempt: 2
I20260620 23:10:43.760566 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.760425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38250)
0620 23:10:43.760492 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.760532 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 23:10:43.760921 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.760836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38262)
0620 23:10:43.760878 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.760902 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:10:43.761036 14491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:41309: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41309: connect: Connection refused (error 111)
I20260620 23:10:43.772955 14498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:43.775234 14498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:43.775389 14498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:43.779352 14498 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.125:41309
I20260620 23:10:43.779408 14565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.125:41309 every 8 connection(s)
I20260620 23:10:43.779744 14498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
I20260620 23:10:43.779927 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14498
I20260620 23:10:43.780032 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-1/wal/instance
I20260620 23:10:43.780221 1153 external_mini_cluster.cc:1428] Setting key 60b189987aefae103873392f65162aec
I20260620 23:10:43.780622 14566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:43.782076 14566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.32.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.124@KRBTEST.COM" created.
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:43.792932 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.782759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.1.32.126:38631)
0620 23:10:43.782819 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.782821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.783027 (+ 206us) server_negotiation.cc:407] Connection header received
0620 23:10:43.783451 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.783452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.783475 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.783526 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.784070 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.784508 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.785436 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.785654 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.785735 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.790096 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.790098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.790098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.790107 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.791568 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.792041 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.792041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.792042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.792087 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.792367 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.792367 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.792368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.792457 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.792512 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.792814 (+ 302us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.792873 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:10:43.793193 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.782719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38631 (local address 127.0.0.1:52194)
0620 23:10:43.782879 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.782935 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.783353 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.783592 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.783595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.783808 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.783982 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.783986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.784605 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.784606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.785341 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.785343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.785744 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.785746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.785862 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.786314 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.786322 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.790013 (+ 3691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.791651 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.791658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.791660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.791968 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.792163 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.792164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.792164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.792294 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.792528 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.792534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.792688 (+ 154us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.792799 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.792956 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260620 23:10:43.793761 14566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:10:43.801627 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.793998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38264)
0620 23:10:43.794038 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.794047 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.794082 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.794327 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.794328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.794473 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.794526 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.794528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.795387 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.795388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.796187 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.796190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.796599 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.796600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.796680 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.796970 (+ 290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.796974 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.799099 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.800760 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.800761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.800761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.800920 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.801124 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.801124 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.801124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.801171 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.801400 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.801401 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.801440 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.801489 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.801572 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 23:10:43.801776 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.794076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.1.32.125:41309)
0620 23:10:43.794198 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.794202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.794219 (+ 17us) server_negotiation.cc:407] Connection header received
0620 23:10:43.794237 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.794238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.794247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.794297 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.794619 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.795298 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.796259 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.796528 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.796573 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.799178 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.799185 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.799185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.799190 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.800697 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.801032 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.801034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.801034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.801076 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.801205 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.801206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.801206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.801327 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.801392 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.801527 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.801636 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260620 23:10:43.802352 14566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:10:43.802670 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.802562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59464)
0620 23:10:43.802608 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.802651 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:10:43.802793 14515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
I20260620 23:10:43.803237 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.803143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59478)
0620 23:10:43.803192 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.803221 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 23:10:43.803346 14566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:43203: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab.
Entry for principal kudu/127.1.32.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab.
I20260620 23:10:43.805297 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:43203
--webserver_interface=127.1.32.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab
--principal=kudu/127.1.32.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:38631,127.1.32.125:41309,127.1.32.124:43203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:10:43.852221 14491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } attempt: 3
I20260620 23:10:43.852617 14566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } attempt: 1
I20260620 23:10:43.853125 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.853001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59482)
0620 23:10:43.853054 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.853092 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:10:43.853446 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.853338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59488)
0620 23:10:43.853402 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.853428 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 23:10:43.853550 14566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:43203: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:10:43.859336 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.852598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38276)
0620 23:10:43.852653 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.852664 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.852702 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.852908 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.852909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.853090 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.853145 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.853147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.853956 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.853957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.854722 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.854725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.855137 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.855139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.855213 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.855584 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.855590 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.856775 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.858271 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.858272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.858273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.858426 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.858720 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.858722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.858722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.858814 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.859107 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.859108 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.859146 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.859201 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.859276 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:10:43.859422 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.852679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.1.32.125:41309)
0620 23:10:43.852738 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.852740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.852749 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:10:43.852774 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.852775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.852786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.852887 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.853250 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.853878 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.854833 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.855071 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.855125 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.856847 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.856848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.856848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.856855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.858197 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.858491 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.858492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.858493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.858550 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.858903 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.858904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.858904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.859037 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.859106 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.859243 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.859319 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:10:43.859838 14491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:10:43.860158 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.860059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59490)
0620 23:10:43.860107 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.860138 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 23:10:43.860491 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.860416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59506)
0620 23:10:43.860455 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.860476 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:10:43.860590 14491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:43203: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
W20260620 23:10:43.883014 14573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:43.883208 14573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:43.883227 14573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:43.884236 14573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:43.884269 14573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:43.884282 14573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:43.884294 14573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:43.884306 14573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:43.885617 14573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:38631,127.1.32.125:41309,127.1.32.124:43203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab
--principal=kudu/127.1.32.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:43203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.32.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:43.885798 14573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:43.885946 14573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:43.890158 14578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:43.890267 14573 server_base.cc:1034] running on GCE node
W20260620 23:10:43.890165 14581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:43.890165 14579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:43.890616 14573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:43.890859 14573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:43.891726 14491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } attempt: 1
I20260620 23:10:43.892035 14573 hybrid_clock.cc:648] HybridClock initialized: now 1781997043892020 us; error 28 us; skew 500 ppm
I20260620 23:10:43.892038 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.891923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59520)
0620 23:10:43.891970 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.892012 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 23:10:43.892336 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.892283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59528)
0620 23:10:43.892305 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.892323 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260620 23:10:43.892436 14491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:43203: Network error: Client connection negotiation failed: client connection to 127.1.32.124:43203: connect: Connection refused (error 111)
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:43.894219 14573 init.cc:377] Logged in from keytab as kudu/127.1.32.124@KRBTEST.COM (short username kudu)
I20260620 23:10:43.894652 14573 webserver.cc:460] Webserver started at http://127.1.32.124:42401/ using document root <none> and password file <none>
I20260620 23:10:43.894937 14573 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:43.895015 14573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:43.895154 14573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:43.896134 14573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data/instance:
uuid: "176af5c0ce824056adfd885949230c6a"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "6087e282ca2bd6ea48deb1754cc90ce6"
server_key_iv: "6a82043b77c3d8c22c1db4d9fa3f2781"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.896503 14573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/wal/instance:
uuid: "176af5c0ce824056adfd885949230c6a"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "6087e282ca2bd6ea48deb1754cc90ce6"
server_key_iv: "6a82043b77c3d8c22c1db4d9fa3f2781"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.897791 14573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:10:43.898391 14588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:43.898530 14573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:10:43.898614 14573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/wal
uuid: "176af5c0ce824056adfd885949230c6a"
format_stamp: "Formatted at 2026-06-20 23:10:43 on dist-test-slave-fzjm"
server_key: "6087e282ca2bd6ea48deb1754cc90ce6"
server_key_iv: "6a82043b77c3d8c22c1db4d9fa3f2781"
server_key_version: "encryptionkey@0"
I20260620 23:10:43.898686 14573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:43.915431 14573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:43.917126 14573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:43.917277 14573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:43.921052 14573 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.124:43203
I20260620 23:10:43.921080 14640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.124:43203 every 8 connection(s)
I20260620 23:10:43.921391 14573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
I20260620 23:10:43.921698 14566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } attempt: 2
I20260620 23:10:43.922353 14641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:43.924363 14641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:43.929877 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14573
I20260620 23:10:43.929984 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996897900784-1153-0/minicluster-data/master-2/wal/instance
I20260620 23:10:43.930182 1153 external_mini_cluster.cc:1428] Setting key 4aadc8a8e001fcc062f49b5f66e326cc
I20260620 23:10:43.931133 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.921983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59544)
0620 23:10:43.922056 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.922068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.922120 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.923139 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.923140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.923265 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.923307 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.923309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.924823 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.924824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.925439 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.925441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.926120 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.926121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.926178 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.926499 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.926504 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.927804 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.929716 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.929717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.929717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.929944 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.930234 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.930236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.930236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.930328 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.930808 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.930809 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.930873 (+ 64us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.930971 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.931063 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 23:10:43.931550 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.922119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.1.32.124:43203)
0620 23:10:43.922318 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.922327 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.922358 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:10:43.922910 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.922939 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.922996 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.923151 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.923406 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.924720 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.925512 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.926048 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.926124 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.927902 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.927909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.927910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.927920 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.929619 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.930056 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.930059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.930059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.930168 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.930395 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.930396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.930396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.930719 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.930838 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.931298 (+ 460us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.931415 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260620 23:10:43.932914 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.924979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38631 (local address 127.0.0.1:52196)
0620 23:10:43.925129 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.925167 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.925248 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.925384 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.925387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.925722 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.925785 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.925787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.926570 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.926571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.927289 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.927292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.927652 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.927653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.927720 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.928150 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.928154 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.929731 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.931835 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.931840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.931842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.932053 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.932260 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.932261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.932261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.932316 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.932575 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.932581 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.932627 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.932721 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.932822 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260620 23:10:43.932928 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.925038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52196 (local address 127.1.32.126:38631)
0620 23:10:43.925089 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.925091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.925238 (+ 147us) server_negotiation.cc:407] Connection header received
0620 23:10:43.925302 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.925302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.925313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.925367 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.925873 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.926467 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.927380 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.927580 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.927647 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.929796 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.929798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.929798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.929805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.931685 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.932144 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.932145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.932145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.932205 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.932390 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.932391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.932391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.932496 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.932582 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.932763 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.932835 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:10:43.933296 14566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d: Bootstrap starting.
I20260620 23:10:43.933432 14641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:10:43.934324 14566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:43.934651 14566 log.cc:824] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:43.935376 14566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d: No bootstrap required, opened a new log
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:10:43.937495 14566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.937642 14566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:43.937680 14566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04386a7525a349b5850f303fce29236d, State: Initialized, Role: FOLLOWER
I20260620 23:10:43.937844 14566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.938091 14648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.938133 14566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:43.938194 14648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:10:43.939425 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.930465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.1.32.124:43203)
0620 23:10:43.930603 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.930605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.930738 (+ 133us) server_negotiation.cc:407] Connection header received
0620 23:10:43.930856 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.930857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.930865 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.930918 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.931644 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.932291 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.933245 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.933494 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.933534 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.936020 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.936022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.936022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.936028 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.938125 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.938516 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.938517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.938517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.938579 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.938834 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.938835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.938835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.938986 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.939117 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.939210 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.939306 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
W20260620 23:10:43.939901 14659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:10:43.940179 14659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:10:43.940629 14498 master_runner.cc:186] Error getting master registration for 127.1.32.126:38631: OK, instance_id { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" instance_seqno: 1781997043626936 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:10:43.943362 14498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:43.944672 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.933724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.1.32.125:41309)
0620 23:10:43.933756 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.933758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.933765 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:10:43.933803 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.933804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.933812 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.933857 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.934311 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.934842 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.939599 (+ 4757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.939855 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.939893 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.941739 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.941740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.941740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.941746 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.943557 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.943913 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.943914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.943914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.943966 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.944163 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.944163 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.944164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.944295 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.944344 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.944517 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.944581 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:10:43.944932 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.933673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41309 (local address 127.0.0.1:38288)
0620 23:10:43.933711 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.933720 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.933752 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.934065 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.934066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.934195 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.934234 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.934235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.934957 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.934958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.939515 (+ 4557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.939518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.939962 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.939963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.940032 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.940400 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.940404 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.941644 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.943639 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.943641 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.943642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.943836 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.944043 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.944043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.944044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.944098 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.944389 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.944390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.944425 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.944766 (+ 341us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.944852 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 23:10:43.945691 14641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:10:43.947248 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.940024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52212 (local address 127.1.32.126:38631)
0620 23:10:43.940054 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.940055 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.940061 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:10:43.940078 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.940079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.940087 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.940134 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.940690 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.941331 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.942271 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.942515 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.942585 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.944794 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.944796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.944797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.944804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.946167 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.946507 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.946508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.946508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.946555 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.946745 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.946746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.946746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.946846 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.947032 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.947087 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.947152 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:10:43.953143 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.946002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.1.32.124:43203)
0620 23:10:43.946028 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.946029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.946076 (+ 47us) server_negotiation.cc:407] Connection header received
0620 23:10:43.946148 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.946148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.946156 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.946209 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.946503 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.947179 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.948078 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.948332 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.948369 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.950118 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.950119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.950120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.950126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.951719 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.952052 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.952052 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.952053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.952093 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.952257 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.952258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.952258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.952743 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.952818 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.952982 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.953055 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260620 23:10:43.953392 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.945880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59554)
0620 23:10:43.946017 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.946028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.946087 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.946212 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.946213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.946368 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.946425 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.946427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.947257 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.947257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.947994 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.947996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.948438 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.948439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.948488 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.948837 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.948847 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.950034 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.951798 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.951798 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.951799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.951984 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.952148 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.952149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.952149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.952191 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.952820 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.952821 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.952870 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.953236 (+ 366us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.953310 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260620 23:10:43.954541 14641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: Bootstrap starting.
I20260620 23:10:43.954790 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.947938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.1.32.125:41309)
0620 23:10:43.947966 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.947967 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.947974 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:10:43.948020 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.948021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.948028 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.948077 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.948482 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.949001 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.949573 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.949774 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.949825 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.952102 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.952103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.952103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.952109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.953734 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.954083 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.954084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.954084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.954141 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.954338 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.954339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.954339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.954449 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.954601 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.954657 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.954722 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 23:10:43.955233 14641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:43.955569 14641 log.cc:824] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:43.956274 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:10:43.956386 14641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: No bootstrap required, opened a new log
I20260620 23:10:43.958823 14641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.959012 14641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:43.959105 14641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176af5c0ce824056adfd885949230c6a, State: Initialized, Role: FOLLOWER
I20260620 23:10:43.959250 14641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.959522 14670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.959576 14641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:43.959657 14670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:10:43.961308 14681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:10:43.961417 14681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:10:43.961858 14573 master_runner.cc:186] Error getting master registration for 127.1.32.126:38631: OK, instance_id { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" instance_seqno: 1781997043626936 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:10:43.964497 14573 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:10:43.966728 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.957639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.1.32.125:41309)
0620 23:10:43.957668 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.957670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.958216 (+ 546us) server_negotiation.cc:407] Connection header received
0620 23:10:43.958244 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.958245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.958256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.958320 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.958976 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.959648 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.960671 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.960937 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.960998 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.962443 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.962444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.962444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.962450 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.964696 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.965966 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.965967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.965967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.966022 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.966231 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.966231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.966232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.966345 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.966432 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.966578 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.966649 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:10:43.967581 14491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } attempt: 2
I20260620 23:10:43.968909 14648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:43.968982 14648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.969235 14648 leader_election.cc:288] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67f65aff26f247c6ba6f61abaac6d7ec (127.1.32.126:38631), 176af5c0ce824056adfd885949230c6a (127.1.32.124:43203)
I20260620 23:10:43.969521 14615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04386a7525a349b5850f303fce29236d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176af5c0ce824056adfd885949230c6a" is_pre_election: true
I20260620 23:10:43.969640 14615 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04386a7525a349b5850f303fce29236d in term 0.
I20260620 23:10:43.969875 14515 leader_election.cc:302] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04386a7525a349b5850f303fce29236d, 176af5c0ce824056adfd885949230c6a; no voters:
I20260620 23:10:43.969993 14648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:10:43.970049 14648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:10:43.970070 14648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:43.970969 14648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.971089 14648 leader_election.cc:288] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [CANDIDATE]: Term 1 election: Requested vote from peers 67f65aff26f247c6ba6f61abaac6d7ec (127.1.32.126:38631), 176af5c0ce824056adfd885949230c6a (127.1.32.124:43203)
I20260620 23:10:43.971282 14615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04386a7525a349b5850f303fce29236d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176af5c0ce824056adfd885949230c6a"
I20260620 23:10:43.971366 14615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:43.972302 14615 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04386a7525a349b5850f303fce29236d in term 1.
I20260620 23:10:43.972508 14515 leader_election.cc:302] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04386a7525a349b5850f303fce29236d, 176af5c0ce824056adfd885949230c6a; no voters:
I20260620 23:10:43.972627 14648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:10:43.972767 14648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [term 1 LEADER]: Becoming Leader. State: Replica: 04386a7525a349b5850f303fce29236d, State: Running, Role: LEADER
I20260620 23:10:43.972851 14648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
Jun 20 23:10:43 dist-test-slave-fzjm krb5kdc[14410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:10:43.973212 14648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04386a7525a349b5850f303fce29236d. Latest consensus state: current_term: 1 leader_uuid: "04386a7525a349b5850f303fce29236d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.973282 14648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:43.973659 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.957611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.1.32.126:38631)
0620 23:10:43.961484 (+ 3873us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.961486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.961493 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:10:43.961514 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.961515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.961524 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.961586 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.963383 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.964014 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.965660 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.965999 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.966065 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.967600 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.967602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.967602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.967609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.971994 (+ 4385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.972458 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.972459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.972460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.972529 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.972773 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.972774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.972774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.972913 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.972998 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.973497 (+ 499us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.973570 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3853}
I20260620 23:10:43.976541 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.968100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:43203 (local address 127.0.0.1:59586)
0620 23:10:43.968142 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:43.968151 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:43.968196 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:43.968339 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:43.968340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:43.968490 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:43.968537 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.968539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.970207 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.970209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.971017 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:43.971020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.971619 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.971620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:43.971675 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.972036 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:43.972041 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:43.973379 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:43.975575 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.975577 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.975577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.975764 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.975948 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:43.975948 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:43.975949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:43.976021 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:43.976249 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:43.976250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:43.976310 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 23:10:43.976374 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:10:43.976453 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 23:10:43.976691 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.968096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.1.32.124:43203)
0620 23:10:43.968134 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.968136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.968182 (+ 46us) server_negotiation.cc:407] Connection header received
0620 23:10:43.968250 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.968250 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.968259 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.968303 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.968630 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.969839 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.971179 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.971461 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.971533 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.973466 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.973485 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.973485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.973492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.975473 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.975835 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.975836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.975836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.975895 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.976073 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.976074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.976074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.976194 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.976244 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.976426 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.976552 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:10:43.978299 14491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: Bootstrap starting.
I20260620 23:10:43.979097 14491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:43.979305 14685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:10:43.979439 14491 log.cc:824] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:43.980397 14491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: No bootstrap required, opened a new log
I20260620 23:10:43.980723 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:43.957928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.1.32.124:43203)
0620 23:10:43.957979 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:43.957981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:43.962748 (+ 4767us) server_negotiation.cc:407] Connection header received
0620 23:10:43.963201 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:43.963202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:43.963212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:43.963243 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:43.968031 (+ 4788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.968744 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.974428 (+ 5684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:43.974711 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:43.974782 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:43.977770 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:43.977772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:43.977772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:43.977779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:43.979441 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.979862 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.979863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.979864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.979918 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:43.980126 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:43.980127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:43.980127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:43.980265 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:43.980370 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:43.980567 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 23:10:43.980641 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:10:43.981156 14685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:10:43.982311 14491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.982419 14491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:43.982455 14491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67f65aff26f247c6ba6f61abaac6d7ec, State: Initialized, Role: FOLLOWER
I20260620 23:10:43.982574 14491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } }
I20260620 23:10:43.982872 14491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:43.982945 14464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04386a7525a349b5850f303fce29236d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67f65aff26f247c6ba6f61abaac6d7ec"
I20260620 23:10:43.983055 14464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:43.983180 14615 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [term 1 FOLLOWER]: Refusing update from remote peer 04386a7525a349b5850f303fce29236d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:43.983485 14683 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [LEADER]: Connected to new peer: Peer: permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:43.983879 14464 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04386a7525a349b5850f303fce29236d in term 1.
I20260620 23:10:43.983996 14688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.984674 14670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04386a7525a349b5850f303fce29236d. Latest consensus state: current_term: 1 leader_uuid: "04386a7525a349b5850f303fce29236d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.984776 14670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:10:43.984117 14688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:10:43.984231 14463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [term 1 FOLLOWER]: Refusing update from remote peer 04386a7525a349b5850f303fce29236d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:43.984004 14465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04386a7525a349b5850f303fce29236d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" is_pre_election: true
I20260620 23:10:43.985466 14465 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 04386a7525a349b5850f303fce29236d in term 1. Re-sending same reply.
I20260620 23:10:43.985452 14683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04386a7525a349b5850f303fce29236d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.985522 14683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:43.985929 14648 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:43.986110 14670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04386a7525a349b5850f303fce29236d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.986178 14670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:10:43.986416 14687 mvcc.cc:205] Tried to move back new op lower bound from 7299059892151492608 to 7299059892113371136. Current Snapshot: MvccSnapshot[applied={T|T < 7299059892151492608}]
I20260620 23:10:43.987936 14688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04386a7525a349b5850f303fce29236d. Latest consensus state: current_term: 1 leader_uuid: "04386a7525a349b5850f303fce29236d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.988021 14688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:10:43.988054 14688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04386a7525a349b5850f303fce29236d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:43.988101 14688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:10:43.990693 14699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:10:43.990746 14699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:10:43.990971 14700 mvcc.cc:205] Tried to move back new op lower bound from 7299059892151492608 to 7299059892113371136. Current Snapshot: MvccSnapshot[applied={T|T < 7299059892151492608 or (T in {7299059892151492608})}]
I20260620 23:10:43.995343 14685 catalog_manager.cc:1347] Generated new cluster ID: 3b3178a3d8c84341844a10e21a7e73b6
I20260620 23:10:43.995409 14685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:10:43.996016 14704 mvcc.cc:205] Tried to move back new op lower bound from 7299059892151492608 to 7299059892113371136. Current Snapshot: MvccSnapshot[applied={T|T < 7299059892151492608}]
I20260620 23:10:44.007926 14648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04386a7525a349b5850f303fce29236d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f65aff26f247c6ba6f61abaac6d7ec" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38631 } } peers { permanent_uuid: "04386a7525a349b5850f303fce29236d" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41309 } } peers { permanent_uuid: "176af5c0ce824056adfd885949230c6a" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 43203 } } }
I20260620 23:10:44.008016 14648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:44.009594 14685 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:10:44.010222 14685 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:10:44.018823 14685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04386a7525a349b5850f303fce29236d: Generated new TSK 0
I20260620 23:10:44.018997 14685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:10:44.965273 14681 catalog_manager.cc:1259] Loaded cluster ID: 3b3178a3d8c84341844a10e21a7e73b6
I20260620 23:10:44.965355 14681 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: loading cluster ID for follower catalog manager: success
I20260620 23:10:44.966740 14681 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: acquiring CA information for follower catalog manager: success
I20260620 23:10:44.967129 14681 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 176af5c0ce824056adfd885949230c6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:10:44.991219 14699 catalog_manager.cc:1259] Loaded cluster ID: 3b3178a3d8c84341844a10e21a7e73b6
I20260620 23:10:44.991303 14699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: loading cluster ID for follower catalog manager: success
I20260620 23:10:44.992558 14699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: acquiring CA information for follower catalog manager: success
I20260620 23:10:44.992908 14699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 67f65aff26f247c6ba6f61abaac6d7ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:10:45.071307 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14423
W20260620 23:10:45.076076 14517 connection.cc:535] client connection to 127.1.32.126:38631 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:10:45.076368 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14498
I20260620 23:10:45.080909 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14573
2026-06-20T23:10:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2677 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:10:45 dist-test-slave-fzjm krb5kdc[14716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:10:45 dist-test-slave-fzjm krb5kdc[14716](info): set up 2 sockets
Jun 20 23:10:45 dist-test-slave-fzjm krb5kdc[14716](info): commencing operation
krb5kdc: starting...
W20260620 23:10:46.094688 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:10:46 dist-test-slave-fzjm krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:10:46.208814 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39745
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:39745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:46.286221 14729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:46.286474 14729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:46.286509 14729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:46.287655 14729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:46.287727 14729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:46.287755 14729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:46.287786 14729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:46.287818 14729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:46.289260 14729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:39745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:46.289536 14729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:46.289767 14729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:46.294670 14734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:46.294718 14737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:46.294718 14735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:46.294749 14729 server_base.cc:1034] running on GCE node
I20260620 23:10:46.295125 14729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:46.295485 14729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:46.296669 14729 hybrid_clock.cc:648] HybridClock initialized: now 1781997046296643 us; error 37 us; skew 500 ppm
Jun 20 23:10:46 dist-test-slave-fzjm krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:46.299119 14729 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:10:46.299575 14729 webserver.cc:460] Webserver started at http://127.1.32.126:42297/ using document root <none> and password file <none>
I20260620 23:10:46.299749 14729 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:46.299791 14729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:46.299885 14729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:46.300864 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "86dca35bfee541819fed74ffab354dd8"
format_stamp: "Formatted at 2026-06-20 23:10:46 on dist-test-slave-fzjm"
server_key: "489d2f9ae65cdbb207608f58bb72d0c7"
server_key_iv: "214db6ce611767d362f40b9d2dcd750e"
server_key_version: "encryptionkey@0"
I20260620 23:10:46.301157 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "86dca35bfee541819fed74ffab354dd8"
format_stamp: "Formatted at 2026-06-20 23:10:46 on dist-test-slave-fzjm"
server_key: "489d2f9ae65cdbb207608f58bb72d0c7"
server_key_iv: "214db6ce611767d362f40b9d2dcd750e"
server_key_version: "encryptionkey@0"
I20260620 23:10:46.302438 14729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:10:46.303161 14744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:46.303305 14729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:10:46.303404 14729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "86dca35bfee541819fed74ffab354dd8"
format_stamp: "Formatted at 2026-06-20 23:10:46 on dist-test-slave-fzjm"
server_key: "489d2f9ae65cdbb207608f58bb72d0c7"
server_key_iv: "214db6ce611767d362f40b9d2dcd750e"
server_key_version: "encryptionkey@0"
I20260620 23:10:46.303457 14729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:46.317963 14729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:46.320313 14729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:46.320431 14729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:46.324105 14729 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:39745
I20260620 23:10:46.324167 14796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:39745 every 8 connection(s)
I20260620 23:10:46.324506 14729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:10:46.325337 14797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:46.327625 14797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8: Bootstrap starting.
I20260620 23:10:46.328222 14797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:46.328490 14797 log.cc:824] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:46.329216 14797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8: No bootstrap required, opened a new log
I20260620 23:10:46.330708 14797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dca35bfee541819fed74ffab354dd8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39745 } }
I20260620 23:10:46.330860 14797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:46.330896 14797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dca35bfee541819fed74ffab354dd8, State: Initialized, Role: FOLLOWER
I20260620 23:10:46.331092 14797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dca35bfee541819fed74ffab354dd8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39745 } }
I20260620 23:10:46.331190 14797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:10:46.331231 14797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:10:46.331280 14797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:46.331768 14797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dca35bfee541819fed74ffab354dd8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39745 } }
I20260620 23:10:46.331894 14797 leader_election.cc:302] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86dca35bfee541819fed74ffab354dd8; no voters:
I20260620 23:10:46.332070 14797 leader_election.cc:288] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:10:46.332111 14800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:10:46.332283 14800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [term 1 LEADER]: Becoming Leader. State: Replica: 86dca35bfee541819fed74ffab354dd8, State: Running, Role: LEADER
I20260620 23:10:46.332324 14797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:46.332365 14800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dca35bfee541819fed74ffab354dd8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39745 } }
I20260620 23:10:46.332957 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14729
I20260620 23:10:46.333061 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:10:46.333235 1153 external_mini_cluster.cc:1428] Setting key 62b705b0cc76f1982d4aa5729158faed
I20260620 23:10:46.332864 14802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86dca35bfee541819fed74ffab354dd8. Latest consensus state: current_term: 1 leader_uuid: "86dca35bfee541819fed74ffab354dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dca35bfee541819fed74ffab354dd8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39745 } } }
I20260620 23:10:46.332741 14801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86dca35bfee541819fed74ffab354dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86dca35bfee541819fed74ffab354dd8" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39745 } } }
W20260620 23:10:46.334003 14815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260620 23:10:46.334232 14802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:46.334240 14801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8 [sys.catalog]: This master's current role is: LEADER
W20260620 23:10:46.334251 14815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:10:46.334774 14819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:10:46.335009 14819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:10:46.336638 14819 catalog_manager.cc:1347] Generated new cluster ID: a26ca6c1062d459d84ec9fd2280036cd
I20260620 23:10:46.336694 14819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:10:46 dist-test-slave-fzjm krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:46.345129 14819 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:10:46.345359 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:46.333932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.1.32.126:39745)
0620 23:10:46.334195 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:46.334205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:46.334244 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:10:46.334380 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:46.334388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:46.334448 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:46.334612 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:46.335472 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:46.336809 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:46.337885 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:46.338503 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:46.338627 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:46.341352 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:46.341355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:46.341356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:46.341367 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:46.343417 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:46.343834 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:46.343837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:46.343837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:46.343951 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:46.344192 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:46.344193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:46.344193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:46.344493 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:46.344676 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:46.344764 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 23:10:46.345018 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
I20260620 23:10:46.345815 14819 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:10:46.346570 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:10:46.347936 14819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86dca35bfee541819fed74ffab354dd8: Generated new TSK 0
I20260620 23:10:46.348080 14819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:10:46.400395 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14729
2026-06-20T23:10:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1318 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:10:46 dist-test-slave-fzjm krb5kdc[14824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:10:46 dist-test-slave-fzjm krb5kdc[14824](info): set up 2 sockets
Jun 20 23:10:46 dist-test-slave-fzjm krb5kdc[14824](info): commencing operation
krb5kdc: starting...
W20260620 23:10:47.426859 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:10:47 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:10:47.535683 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45571
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:45571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:47.612100 14837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:47.612279 14837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:47.612299 14837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:47.613348 14837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:47.613381 14837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:47.613395 14837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:47.613407 14837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:47.613420 14837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:47.614773 14837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:45571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:47.615042 14837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:47.615207 14837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:47.619952 14843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:47.620009 14842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:47.619935 14845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:47.619940 14837 server_base.cc:1034] running on GCE node
I20260620 23:10:47.620342 14837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:47.620631 14837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:47.621835 14837 hybrid_clock.cc:648] HybridClock initialized: now 1781997047621820 us; error 45 us; skew 500 ppm
Jun 20 23:10:47 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997047, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:47.624333 14837 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:10:47.624742 14837 webserver.cc:460] Webserver started at http://127.1.32.126:41869/ using document root <none> and password file <none>
I20260620 23:10:47.624964 14837 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:47.625031 14837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:47.625176 14837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:47.626173 14837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "8656af006d9f489da448d56f56926c59"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "3be30ba469ad15044e51a76743fce8f9"
server_key_iv: "21455c8ea2452e2dfb9174ca893e5bc1"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.626528 14837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "8656af006d9f489da448d56f56926c59"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "3be30ba469ad15044e51a76743fce8f9"
server_key_iv: "21455c8ea2452e2dfb9174ca893e5bc1"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.627920 14837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:10:47.628638 14852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.628811 14837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:10:47.628919 14837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "8656af006d9f489da448d56f56926c59"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "3be30ba469ad15044e51a76743fce8f9"
server_key_iv: "21455c8ea2452e2dfb9174ca893e5bc1"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.629016 14837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:47.643297 14837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:47.645040 14837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:47.645197 14837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:47.648839 14837 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:45571
I20260620 23:10:47.648869 14904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:45571 every 8 connection(s)
I20260620 23:10:47.649170 14837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:10:47.649936 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14837
I20260620 23:10:47.650038 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:10:47.650077 14905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:47.650234 1153 external_mini_cluster.cc:1428] Setting key 11c9218e43873f2e647b8d4d69d6c2d3
I20260620 23:10:47.653148 14905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Bootstrap starting.
I20260620 23:10:47.653852 14905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:47.654160 14905 log.cc:824] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:47.654853 14905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: No bootstrap required, opened a new log
I20260620 23:10:47.656744 14905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
Jun 20 23:10:47 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:47.656917 14905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:47.656952 14905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8656af006d9f489da448d56f56926c59, State: Initialized, Role: FOLLOWER
I20260620 23:10:47.657078 14905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:47.657176 14905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:10:47.657208 14905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:10:47.657253 14905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:47.657797 14905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:47.657958 14905 leader_election.cc:302] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8656af006d9f489da448d56f56926c59; no voters:
I20260620 23:10:47.658111 14905 leader_election.cc:288] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:10:47.658154 14910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:10:47.658305 14910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 LEADER]: Becoming Leader. State: Replica: 8656af006d9f489da448d56f56926c59, State: Running, Role: LEADER
I20260620 23:10:47.658396 14910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:47.658442 14905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:47.658671 14910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8656af006d9f489da448d56f56926c59. Latest consensus state: current_term: 1 leader_uuid: "8656af006d9f489da448d56f56926c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } } }
I20260620 23:10:47.658684 14911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8656af006d9f489da448d56f56926c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } } }
I20260620 23:10:47.658875 14911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:47.658787 14910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: This master's current role is: LEADER
W20260620 23:10:47.660151 14926 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:10:47.660344 14926 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:10:47.660419 14917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:10:47.660619 14917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:10:47.661654 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:47.650770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.1.32.126:45571)
0620 23:10:47.650959 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:47.650967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:47.650990 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:10:47.651463 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:47.651469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:47.651512 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:47.651618 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:47.652364 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.653332 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.654391 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.654866 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.655032 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:47.657466 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:47.657469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:47.657469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:47.657480 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:47.659918 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:47.660241 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:47.660244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:47.660244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:47.660346 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:47.660560 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:47.660561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:47.660561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:47.660880 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:47.661373 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:47.661437 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 23:10:47.661530 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260620 23:10:47.662230 14917 catalog_manager.cc:1347] Generated new cluster ID: 67fbe18465be41d4b3001d61fb5137b1
I20260620 23:10:47.662281 14917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:10:47.671743 14917 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:10:47.672235 14917 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:10:47.677505 14917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Generated new TSK 0
I20260620 23:10:47.677673 14917 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:10:47.701346 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45571
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:47.777949 14931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:47.778138 14931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:47.778162 14931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:47.779294 14931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:47.779330 14931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:47.779356 14931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:10:47.781051 14931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:47.781266 14931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:47.781479 14931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:47.786093 14937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:47.786172 14931 server_base.cc:1034] running on GCE node
W20260620 23:10:47.786094 14939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:47.786099 14936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:47.786487 14931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:47.786666 14931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:47.787799 14931 hybrid_clock.cc:648] HybridClock initialized: now 1781997047787787 us; error 28 us; skew 500 ppm
Jun 20 23:10:47 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997047, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:47.790112 14931 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:10:47.790490 14931 webserver.cc:460] Webserver started at http://127.1.32.65:38535/ using document root <none> and password file <none>
I20260620 23:10:47.790650 14931 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:47.790688 14931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:47.790768 14931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:47.791742 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "287afdcbfc8e447aac28c912edef9184"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "940276e3ac0e400b78cf2debdafed32d"
server_key_iv: "65b45c138c4aa43c4ce4458a3a2cfb9d"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.792030 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "287afdcbfc8e447aac28c912edef9184"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "940276e3ac0e400b78cf2debdafed32d"
server_key_iv: "65b45c138c4aa43c4ce4458a3a2cfb9d"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.793169 14931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:10:47.793798 14946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.793965 14931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:10:47.794059 14931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "287afdcbfc8e447aac28c912edef9184"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "940276e3ac0e400b78cf2debdafed32d"
server_key_iv: "65b45c138c4aa43c4ce4458a3a2cfb9d"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.794131 14931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:47.809386 14931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:47.811044 14931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:47.811187 14931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:47.811384 14931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:47.811694 14931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:47.811743 14931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.811771 14931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:47.811796 14931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.817201 14931 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:46449
I20260620 23:10:47.817266 15059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:46449 every 8 connection(s)
I20260620 23:10:47.817541 14931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:10:47 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997047, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:47.825263 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 14931
I20260620 23:10:47.825388 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:10:47.825577 1153 external_mini_cluster.cc:1428] Setting key be285cc986246a2152e507c1f0d4f907
I20260620 23:10:47.826392 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:47.818161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:40643 (local address 127.1.32.126:45571)
0620 23:10:47.818204 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:47.818206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:47.818399 (+ 193us) server_negotiation.cc:407] Connection header received
0620 23:10:47.818869 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:47.818870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:47.818882 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:47.818957 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:47.820027 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.820535 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.821461 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.821663 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.821743 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:47.823708 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:47.823710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:47.823710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:47.823717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:47.824914 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:47.825305 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:47.825306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:47.825307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:47.825389 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:47.825636 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:47.825637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:47.825638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:47.825793 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:47.825904 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:47.826201 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 23:10:47.826301 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:10:47.826512 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:47.818129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:40643)
0620 23:10:47.818251 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:47.818296 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:47.818757 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:47.819028 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:47.819031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:47.819681 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:47.819927 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:47.819932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.820624 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.820625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:47.821372 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:47.821374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.821754 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.821755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:47.821877 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:47.822254 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:47.822260 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:47.823612 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:47.825013 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:47.825019 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:47.825021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:47.825228 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:47.825460 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:47.825460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:47.825461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:47.825565 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:47.825915 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:47.825921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:47.826064 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 23:10:47.826167 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 23:10:47.826323 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260620 23:10:47.828027 15060 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45571
I20260620 23:10:47.828127 15060 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:47.828308 15060 heartbeater.cc:511] Master 127.1.32.126:45571 requested a full tablet report, sending...
I20260620 23:10:47.828924 14869 ts_manager.cc:194] Registered new tserver with Master: 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449)
I20260620 23:10:47.829809 14869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:40643
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:10:47.847330 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45571
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:47.924220 15065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:47.924405 15065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:47.924427 15065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:47.925524 15065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:47.925560 15065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:47.925586 15065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:10:47.926940 15065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:47.927100 15065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:47.927259 15065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:47.931875 15070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:47.931882 15073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:47.932077 15065 server_base.cc:1034] running on GCE node
W20260620 23:10:47.931900 15071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:47.932385 15065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:47.932657 15065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:47.933816 15065 hybrid_clock.cc:648] HybridClock initialized: now 1781997047933784 us; error 44 us; skew 500 ppm
Jun 20 23:10:47 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997047, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:47.936332 15065 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:10:47.936770 15065 webserver.cc:460] Webserver started at http://127.1.32.66:43087/ using document root <none> and password file <none>
I20260620 23:10:47.937005 15065 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:47.937068 15065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:47.937203 15065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:47.938134 15065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "820d54f23d2a4dee9b7f4b17c1808ebb"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "c2077d519e2d709c4c99582006a8f472"
server_key_iv: "d8115d93e4da95e853406e857272b885"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.938470 15065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "820d54f23d2a4dee9b7f4b17c1808ebb"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "c2077d519e2d709c4c99582006a8f472"
server_key_iv: "d8115d93e4da95e853406e857272b885"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.939829 15065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:10:47.940533 15080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.940697 15065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.940788 15065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "820d54f23d2a4dee9b7f4b17c1808ebb"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "c2077d519e2d709c4c99582006a8f472"
server_key_iv: "d8115d93e4da95e853406e857272b885"
server_key_version: "encryptionkey@0"
I20260620 23:10:47.940872 15065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:47.962499 15065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:47.964267 15065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:47.964426 15065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:47.964638 15065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:47.964921 15065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:47.964983 15065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.965015 15065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:47.965041 15065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:47.970535 15065 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:37509
I20260620 23:10:47.970580 15193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:37509 every 8 connection(s)
I20260620 23:10:47.970875 15065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:10:47.971552 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15065
I20260620 23:10:47.971664 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:10:47.971853 1153 external_mini_cluster.cc:1428] Setting key e82d577bb4075ab666b3720a2c82de58
Jun 20 23:10:47 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997047, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:47.980800 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:47.971560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:33355 (local address 127.1.32.126:45571)
0620 23:10:47.971618 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:47.971620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:47.972185 (+ 565us) server_negotiation.cc:407] Connection header received
0620 23:10:47.972777 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:47.972778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:47.972790 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:47.972845 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:47.973683 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.974138 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.975175 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.975468 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.975529 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:47.977955 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:47.977957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:47.977957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:47.977964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:47.979557 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:47.979913 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:47.979914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:47.979915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:47.979974 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:47.980199 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:47.980199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:47.980200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:47.980332 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:47.980401 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:47.980625 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 23:10:47.980700 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:10:47.980818 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:47.971546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:33355)
0620 23:10:47.972053 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:47.972098 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:47.972680 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:47.972890 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:47.972893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:47.973397 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:47.973588 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:47.973593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.974226 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.974227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:47.975087 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:47.975089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:47.975560 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:47.975561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:47.975665 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:47.976210 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:47.976217 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:47.977869 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:47.979633 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:47.979639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:47.979641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:47.979840 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:47.980041 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:47.980041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:47.980041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:47.980137 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:47.980396 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:47.980399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:47.980503 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 23:10:47.980576 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:10:47.980679 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
I20260620 23:10:47.981285 15194 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45571
I20260620 23:10:47.981370 15194 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:47.981544 15194 heartbeater.cc:511] Master 127.1.32.126:45571 requested a full tablet report, sending...
I20260620 23:10:47.981946 14869 ts_manager.cc:194] Registered new tserver with Master: 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509)
I20260620 23:10:47.982292 14869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:33355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:10:47.995633 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45571
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:48.072569 15199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:48.072775 15199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:48.072809 15199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:48.073877 15199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:48.073961 15199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:48.074008 15199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:10:48.075500 15199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:48.075737 15199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:48.075947 15199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:48.080540 15204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.080572 15207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.080575 15205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:48.080817 15199 server_base.cc:1034] running on GCE node
I20260620 23:10:48.081036 15199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:48.081257 15199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:48.082389 15199 hybrid_clock.cc:648] HybridClock initialized: now 1781997048082372 us; error 40 us; skew 500 ppm
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:48.084794 15199 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:10:48.085263 15199 webserver.cc:460] Webserver started at http://127.1.32.67:40145/ using document root <none> and password file <none>
I20260620 23:10:48.085487 15199 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:48.085570 15199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:48.085711 15199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:10:48.086661 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "4fc3e3c1b42947aa919116e60bf330cc"
format_stamp: "Formatted at 2026-06-20 23:10:48 on dist-test-slave-fzjm"
server_key: "1592ae6e7dba1111e76db0620625ce87"
server_key_iv: "8e49e072a5761037b31364604df52244"
server_key_version: "encryptionkey@0"
I20260620 23:10:48.087044 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "4fc3e3c1b42947aa919116e60bf330cc"
format_stamp: "Formatted at 2026-06-20 23:10:48 on dist-test-slave-fzjm"
server_key: "1592ae6e7dba1111e76db0620625ce87"
server_key_iv: "8e49e072a5761037b31364604df52244"
server_key_version: "encryptionkey@0"
I20260620 23:10:48.088351 15199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:10:48.089079 15214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.089257 15199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:10:48.089350 15199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "4fc3e3c1b42947aa919116e60bf330cc"
format_stamp: "Formatted at 2026-06-20 23:10:48 on dist-test-slave-fzjm"
server_key: "1592ae6e7dba1111e76db0620625ce87"
server_key_iv: "8e49e072a5761037b31364604df52244"
server_key_version: "encryptionkey@0"
I20260620 23:10:48.089433 15199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:48.103302 15199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:48.104938 15199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:48.105108 15199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:48.105319 15199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:48.105603 15199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:48.105655 15199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.105687 15199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:48.105712 15199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.111279 15199 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:34437
I20260620 23:10:48.111327 15327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:34437 every 8 connection(s)
I20260620 23:10:48.111634 15199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:48.119771 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15199
I20260620 23:10:48.119792 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.112312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:44861 (local address 127.1.32.126:45571)
0620 23:10:48.112376 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.112378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.112562 (+ 184us) server_negotiation.cc:407] Connection header received
0620 23:10:48.112976 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.112977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.112985 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.113041 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:48.113852 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.114329 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.115300 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.115482 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.115534 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.117360 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:48.117361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:48.117362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:48.117367 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:48.118509 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.118854 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.118854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.118855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.118988 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.119240 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.119240 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.119240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.119342 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:48.119393 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.119663 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.119732 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 23:10:48.119877 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:10:48.120061 1153 external_mini_cluster.cc:1428] Setting key 3fb8844457903b3bcd479a482c0fe4ad
I20260620 23:10:48.120088 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.112267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:44861)
0620 23:10:48.112443 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.112480 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.112902 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.113090 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.113093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.113614 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:48.113782 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.113786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.114402 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.114403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.115231 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.115233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.115551 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.115551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.115640 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.116040 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:48.116047 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:48.117272 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:48.118575 (+ 1303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.118580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.118582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.118789 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.119074 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.119075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.119075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.119177 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.119399 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:48.119418 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:48.119518 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.119627 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.119793 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":113,"threads_started":1}
I20260620 23:10:48.120570 15328 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45571
I20260620 23:10:48.120661 15328 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:48.120815 15328 heartbeater.cc:511] Master 127.1.32.126:45571 requested a full tablet report, sending...
I20260620 23:10:48.121160 14868 ts_manager.cc:194] Registered new tserver with Master: 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
I20260620 23:10:48.121491 14868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:44861
I20260620 23:10:48.131529 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:10:48.138415 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.132843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.1.32.126:45571)
0620 23:10:48.132896 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.132899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.133075 (+ 176us) server_negotiation.cc:407] Connection header received
0620 23:10:48.133108 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.133109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.133116 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.133164 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:48.133717 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.134163 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.134848 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.135151 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.135208 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.136287 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:48.136288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:48.136288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:48.136294 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:48.137595 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.137848 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.137848 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.137849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.137891 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.138018 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.138018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.138018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.138119 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:48.138180 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.138258 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.138306 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:10:48.139976 14868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:10:48.141487 14868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:10:48.147713 15338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.143740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:37509 (local address 127.0.0.1:44562)
0620 23:10:48.144146 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.144155 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.144235 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.144497 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.144500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.144503 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.144573 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.144575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.145481 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.145483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.146656 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.146658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.147441 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.147442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.147514 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.147526 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.147595 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.147642 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":9,"thread_start_us":62,"threads_started":1}
I20260620 23:10:48.147723 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.143923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.1.32.66:37509)
0620 23:10:48.144101 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.144105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.144206 (+ 101us) server_negotiation.cc:407] Connection header received
0620 23:10:48.144305 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.144307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.144323 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.144405 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.144637 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.145387 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.146736 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.147342 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.147434 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.147493 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.147608 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.147648 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260620 23:10:48.148123 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.143748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:34437 (local address 127.0.0.1:59194)
0620 23:10:48.144413 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.144422 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.144464 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.144724 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.144725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.144727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.144764 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.144765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.145934 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.145935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.146960 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.146962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.147872 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.147873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.147936 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.147943 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.147999 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.148033 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":4,"thread_start_us":75,"threads_started":1}
I20260620 23:10:48.148132 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.143737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46449 (local address 127.0.0.1:51274)
0620 23:10:48.144079 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.144118 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.144197 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.144497 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.144500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.144504 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.144573 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.144576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.145862 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.145863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.146922 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.146924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.147876 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.147877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.147936 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.147941 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.147997 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.148034 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":6,"thread_start_us":59,"threads_started":1}
I20260620 23:10:48.148173 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.143907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51274 (local address 127.1.32.65:46449)
0620 23:10:48.144081 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.144086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.144192 (+ 106us) server_negotiation.cc:407] Connection header received
0620 23:10:48.144305 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.144307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.144349 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.144417 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.144667 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.145747 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.146984 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.147719 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.147789 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.147847 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.148025 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.148078 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260620 23:10:48.148417 15341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.143948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.1.32.67:34437)
0620 23:10:48.144143 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.144148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.144522 (+ 374us) server_negotiation.cc:407] Connection header received
0620 23:10:48.144566 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.144568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.144583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.144649 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.144849 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.145853 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.147024 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.147719 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.147788 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.147836 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.148265 (+ 429us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.148308 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260620 23:10:48.148725 15128 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4f0a9619d342568bb26ba48e941134 (DEFAULT_TABLE table=test-table [id=9ff8a8f410064a268e4b532a14612400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:48.148725 14994 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4f0a9619d342568bb26ba48e941134 (DEFAULT_TABLE table=test-table [id=9ff8a8f410064a268e4b532a14612400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:48.149087 14994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4f0a9619d342568bb26ba48e941134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:48.149133 15128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4f0a9619d342568bb26ba48e941134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:48.152316 15343 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Bootstrap starting.
I20260620 23:10:48.152652 15262 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4f0a9619d342568bb26ba48e941134 (DEFAULT_TABLE table=test-table [id=9ff8a8f410064a268e4b532a14612400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:48.152913 15343 tablet_bootstrap.cc:654] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:48.152932 15262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4f0a9619d342568bb26ba48e941134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:48.153427 15343 log.cc:824] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:48.153859 15342 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Bootstrap starting.
I20260620 23:10:48.154385 15342 tablet_bootstrap.cc:654] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:48.154844 15342 log.cc:824] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:48.155354 15343 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: No bootstrap required, opened a new log
I20260620 23:10:48.155438 15343 ts_tablet_manager.cc:1397] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:10:48.155624 15342 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: No bootstrap required, opened a new log
I20260620 23:10:48.155686 15342 ts_tablet_manager.cc:1397] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:10:48.155865 15346 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Bootstrap starting.
I20260620 23:10:48.156580 15346 tablet_bootstrap.cc:654] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:48.157023 15346 log.cc:824] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:48.157464 15343 raft_consensus.cc:348] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.157655 15343 raft_consensus.cc:374] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:48.157608 15342 raft_consensus.cc:348] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.157764 15346 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: No bootstrap required, opened a new log
I20260620 23:10:48.157809 15342 raft_consensus.cc:374] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:48.157840 15342 raft_consensus.cc:729] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287afdcbfc8e447aac28c912edef9184, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.157703 15343 raft_consensus.cc:729] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820d54f23d2a4dee9b7f4b17c1808ebb, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.157884 15346 ts_tablet_manager.cc:1397] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:10:48.157963 15343 consensus_queue.cc:260] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.157963 15342 consensus_queue.cc:260] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.158245 15342 ts_tablet_manager.cc:1428] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 23:10:48.158264 15343 ts_tablet_manager.cc:1428] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:10:48.158360 15194 heartbeater.cc:503] Master 127.1.32.126:45571 was elected leader, sending a full tablet report...
I20260620 23:10:48.158365 15060 heartbeater.cc:503] Master 127.1.32.126:45571 was elected leader, sending a full tablet report...
I20260620 23:10:48.160367 15346 raft_consensus.cc:348] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.160509 15346 raft_consensus.cc:374] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:48.160552 15346 raft_consensus.cc:729] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc3e3c1b42947aa919116e60bf330cc, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.160656 15346 consensus_queue.cc:260] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.160923 15346 ts_tablet_manager.cc:1428] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 23:10:48.160989 15328 heartbeater.cc:503] Master 127.1.32.126:45571 was elected leader, sending a full tablet report...
W20260620 23:10:48.221421 15195 tablet.cc:2369] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:10:48.318280 15061 tablet.cc:2369] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:10:48.362192 15329 tablet.cc:2369] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:10:48.406538 15348 raft_consensus.cc:482] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:48.406672 15348 raft_consensus.cc:504] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.407073 15348 leader_election.cc:288] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449), 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
I20260620 23:10:48.411029 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.407410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:59659 (local address 127.1.32.65:46449)
0620 23:10:48.407461 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.407463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.407474 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:10:48.407558 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.407559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.407575 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.407622 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.407836 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.408555 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.410139 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.410639 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.410712 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.410724 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.410923 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.410962 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 23:10:48.411029 15341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.407399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:34323 (local address 127.1.32.67:34437)
0620 23:10:48.407444 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.407446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.407685 (+ 239us) server_negotiation.cc:407] Connection header received
0620 23:10:48.407753 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.407754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.407763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.407815 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.407947 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.408611 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.410139 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.410682 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.410737 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.410746 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.410923 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.410961 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:10:48.411055 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.407279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46449 (local address 127.1.32.66:59659)
0620 23:10:48.407363 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.407376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.407464 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.407661 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.407662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.407664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.407751 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.407753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.408674 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.408676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.410035 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.410039 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.410742 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.410743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.410832 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.410836 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.410920 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.410965 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":69}
I20260620 23:10:48.411065 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.407279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:34437 (local address 127.1.32.66:34323)
0620 23:10:48.407575 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.407585 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.407690 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.407820 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.407820 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.407822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.407879 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.407884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.408681 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.408681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.410036 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.410039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.410745 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.410746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.410831 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.410837 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.410920 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.410964 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":126,"thread_start_us":135,"threads_started":1}
I20260620 23:10:48.411463 15014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f0a9619d342568bb26ba48e941134" candidate_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "287afdcbfc8e447aac28c912edef9184" is_pre_election: true
I20260620 23:10:48.411464 15282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f0a9619d342568bb26ba48e941134" candidate_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc3e3c1b42947aa919116e60bf330cc" is_pre_election: true
I20260620 23:10:48.411633 15014 raft_consensus.cc:2454] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 820d54f23d2a4dee9b7f4b17c1808ebb in term 0.
I20260620 23:10:48.411638 15282 raft_consensus.cc:2454] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 820d54f23d2a4dee9b7f4b17c1808ebb in term 0.
I20260620 23:10:48.411875 15081 leader_election.cc:302] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 287afdcbfc8e447aac28c912edef9184, 820d54f23d2a4dee9b7f4b17c1808ebb; no voters:
I20260620 23:10:48.412003 15348 raft_consensus.cc:2781] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:10:48.412087 15348 raft_consensus.cc:482] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:10:48.412122 15348 raft_consensus.cc:3037] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:48.412927 15348 raft_consensus.cc:504] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.413085 15348 leader_election.cc:288] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [CANDIDATE]: Term 1 election: Requested vote from peers 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449), 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
I20260620 23:10:48.413287 15282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f0a9619d342568bb26ba48e941134" candidate_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc3e3c1b42947aa919116e60bf330cc"
I20260620 23:10:48.413287 15014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f0a9619d342568bb26ba48e941134" candidate_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "287afdcbfc8e447aac28c912edef9184"
I20260620 23:10:48.413374 15282 raft_consensus.cc:3037] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:48.413383 15014 raft_consensus.cc:3037] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:48.414258 15282 raft_consensus.cc:2454] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 820d54f23d2a4dee9b7f4b17c1808ebb in term 1.
I20260620 23:10:48.414268 15014 raft_consensus.cc:2454] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 820d54f23d2a4dee9b7f4b17c1808ebb in term 1.
I20260620 23:10:48.414435 15081 leader_election.cc:302] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 287afdcbfc8e447aac28c912edef9184, 820d54f23d2a4dee9b7f4b17c1808ebb; no voters:
I20260620 23:10:48.414517 15348 raft_consensus.cc:2781] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:10:48.414664 15348 raft_consensus.cc:686] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 LEADER]: Becoming Leader. State: Replica: 820d54f23d2a4dee9b7f4b17c1808ebb, State: Running, Role: LEADER
I20260620 23:10:48.414764 15348 consensus_queue.cc:237] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.415597 14868 catalog_manager.cc:5495] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb reported cstate change: term changed from 0 to 1, leader changed from <none> to 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66). New cstate: current_term: 1 leader_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } health_report { overall_health: UNKNOWN } } }
I20260620 23:10:48.436028 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.433847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.1.32.66:37509)
0620 23:10:48.434001 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.434003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.434010 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:10:48.434047 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.434047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.434062 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.434117 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:10:48.434242 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.434663 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.435411 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.435576 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.435623 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.435739 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:10:48.435828 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:10:48.435882 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.435909 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.435924 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260620 23:10:48.437825 15014 raft_consensus.cc:1261] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Refusing update from remote peer 820d54f23d2a4dee9b7f4b17c1808ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:48.437803 15282 raft_consensus.cc:1261] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 FOLLOWER]: Refusing update from remote peer 820d54f23d2a4dee9b7f4b17c1808ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:48.438162 15352 consensus_queue.cc:1035] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:48.438290 15348 consensus_queue.cc:1035] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:48.442413 14869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:10:48.442487 14869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:10:48.443202 14869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Sending DeleteTablet for 3 replicas of tablet 8a4f0a9619d342568bb26ba48e941134
I20260620 23:10:48.443516 15128 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4f0a9619d342568bb26ba48e941134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:48 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:44562
I20260620 23:10:48.443518 15262 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4f0a9619d342568bb26ba48e941134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:48 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:59194
I20260620 23:10:48.443540 14994 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4f0a9619d342568bb26ba48e941134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:48 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:51274
I20260620 23:10:48.443744 15366 tablet_replica.cc:330] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: stopping tablet replica
I20260620 23:10:48.443796 15364 tablet_replica.cc:330] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: stopping tablet replica
I20260620 23:10:48.443806 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14931
I20260620 23:10:48.443846 15366 raft_consensus.cc:2229] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:10:48.443907 15365 tablet_replica.cc:330] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: stopping tablet replica
I20260620 23:10:48.443953 15366 raft_consensus.cc:2258] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:48.443974 15365 raft_consensus.cc:2229] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:10:48.444062 15365 raft_consensus.cc:2258] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:48.444167 15364 raft_consensus.cc:2229] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:10:48.444253 15364 raft_consensus.cc:2258] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:48.444710 15364 ts_tablet_manager.cc:1905] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:10:48.445315 15365 ts_tablet_manager.cc:1905] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:10:48.446362 15364 ts_tablet_manager.cc:1918] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:10:48.446416 15364 log.cc:1196] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/8a4f0a9619d342568bb26ba48e941134
I20260620 23:10:48.446641 15364 ts_tablet_manager.cc:1939] T 8a4f0a9619d342568bb26ba48e941134 P 820d54f23d2a4dee9b7f4b17c1808ebb: Deleting consensus metadata
I20260620 23:10:48.446797 15365 ts_tablet_manager.cc:1918] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:10:48.446859 15365 log.cc:1196] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/8a4f0a9619d342568bb26ba48e941134
I20260620 23:10:48.447115 15365 ts_tablet_manager.cc:1939] T 8a4f0a9619d342568bb26ba48e941134 P 4fc3e3c1b42947aa919116e60bf330cc: Deleting consensus metadata
I20260620 23:10:48.447402 14856 catalog_manager.cc:4841] TS 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437): tablet 8a4f0a9619d342568bb26ba48e941134 (table test-table [id=9ff8a8f410064a268e4b532a14612400]) successfully deleted
I20260620 23:10:48.450574 14854 catalog_manager.cc:4841] TS 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509): tablet 8a4f0a9619d342568bb26ba48e941134 (table test-table [id=9ff8a8f410064a268e4b532a14612400]) successfully deleted
W20260620 23:10:48.450976 14853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:46449 (error 108)
I20260620 23:10:48.451216 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 15065
I20260620 23:10:48.451380 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.451252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46449 (local address 127.0.0.1:51284)
0620 23:10:48.451315 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.451345 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 23:10:48.455348 14853 catalog_manager.cc:4584] TS 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a4f0a9619d342568bb26ba48e941134: Network error: Client connection negotiation failed: client connection to 127.1.32.65:46449: connect: Connection refused (error 111)
I20260620 23:10:48.456095 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 15199
I20260620 23:10:48.460347 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 14837
I20260620 23:10:48.466005 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45571
--webserver_interface=127.1.32.126
--webserver_port=41869
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:45571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:48.542577 15368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:48.542795 15368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:48.542816 15368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:48.543964 15368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:48.544005 15368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:48.544019 15368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:48.544056 15368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:48.544080 15368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:48.545430 15368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:45571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=41869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:48.545677 15368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:48.545898 15368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:48.554051 15374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.554082 15375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.554082 15377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:48.554237 15368 server_base.cc:1034] running on GCE node
I20260620 23:10:48.554490 15368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:48.554793 15368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:48.555954 15368 hybrid_clock.cc:648] HybridClock initialized: now 1781997048555954 us; error 45 us; skew 500 ppm
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:48.558386 15368 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:10:48.558799 15368 webserver.cc:460] Webserver started at http://127.1.32.126:41869/ using document root <none> and password file <none>
I20260620 23:10:48.559070 15368 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:48.559149 15368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:48.560477 15368 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:48.561117 15384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.561275 15368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:10:48.561393 15368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "8656af006d9f489da448d56f56926c59"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "3be30ba469ad15044e51a76743fce8f9"
server_key_iv: "21455c8ea2452e2dfb9174ca893e5bc1"
server_key_version: "encryptionkey@0"
I20260620 23:10:48.561756 15368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:48.583743 15368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:48.586086 15368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:48.586241 15368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:48.590215 15368 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:45571
I20260620 23:10:48.590260 15436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:45571 every 8 connection(s)
I20260620 23:10:48.590660 15368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:10:48.591068 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15368
I20260620 23:10:48.591408 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:46449
--local_ip_for_outbound_sockets=127.1.32.65
--tserver_master_addrs=127.1.32.126:45571
--webserver_port=38535
--webserver_interface=127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:10:48.592383 15437 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:10:48.593046 15437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Bootstrap starting.
I20260620 23:10:48.601233 15437 log.cc:824] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:48.603175 15437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:10:48.603484 15437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Bootstrap complete.
I20260620 23:10:48.605778 15437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:48.606030 15437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8656af006d9f489da448d56f56926c59, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.606251 15437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:48.606344 15437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:10:48.606398 15437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:10:48.606451 15437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:10:48.607474 15437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:48.607589 15437 leader_election.cc:302] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8656af006d9f489da448d56f56926c59; no voters:
I20260620 23:10:48.607760 15437 leader_election.cc:288] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:10:48.607828 15441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:10:48.607957 15441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 LEADER]: Becoming Leader. State: Replica: 8656af006d9f489da448d56f56926c59, State: Running, Role: LEADER
I20260620 23:10:48.608026 15437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:48.608053 15441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:48.608299 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8656af006d9f489da448d56f56926c59. Latest consensus state: current_term: 2 leader_uuid: "8656af006d9f489da448d56f56926c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } } }
I20260620 23:10:48.608384 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:48.608307 15442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8656af006d9f489da448d56f56926c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } } }
I20260620 23:10:48.608559 15446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:10:48.608580 15442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:48.609273 15446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:10:48.609396 15446 catalog_manager.cc:1259] Loaded cluster ID: 67fbe18465be41d4b3001d61fb5137b1
I20260620 23:10:48.609429 15446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:10:48.609652 15446 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 23:10:48.614005 15446 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 23:10:48.616568 15446 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:10:48.616755 15446 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Loaded TSK: 0
I20260620 23:10:48.616987 15446 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:10:48.674345 15439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:48.674535 15439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:48.674556 15439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:48.675777 15439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:48.675814 15439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:48.675845 15439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:10:48.677209 15439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:46449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=38535
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:48.677426 15439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:48.677657 15439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:48.682374 15464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.682330 15467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:48.682477 15439 server_base.cc:1034] running on GCE node
W20260620 23:10:48.682592 15465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:48.682808 15439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:48.683131 15439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:48.684301 15439 hybrid_clock.cc:648] HybridClock initialized: now 1781997048684282 us; error 36 us; skew 500 ppm
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:48.686776 15439 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:10:48.687273 15439 webserver.cc:460] Webserver started at http://127.1.32.65:38535/ using document root <none> and password file <none>
I20260620 23:10:48.687503 15439 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:48.687556 15439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:48.688834 15439 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:10:48.689523 15474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.689744 15439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:10:48.689862 15439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "287afdcbfc8e447aac28c912edef9184"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "940276e3ac0e400b78cf2debdafed32d"
server_key_iv: "65b45c138c4aa43c4ce4458a3a2cfb9d"
server_key_version: "encryptionkey@0"
I20260620 23:10:48.690186 15439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:48.702680 15439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:48.704665 15439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:48.704806 15439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:48.705024 15439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:48.705454 15481 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:10:48.706785 15439 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:10:48.706836 15439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260620 23:10:48.706866 15439 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:10:48.707463 15439 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:10:48.707500 15439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:10:48.707542 15481 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Bootstrap starting.
I20260620 23:10:48.715092 15439 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:46449
I20260620 23:10:48.715577 15439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:10:48.716194 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15439
I20260620 23:10:48.716535 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:37509
--local_ip_for_outbound_sockets=127.1.32.66
--tserver_master_addrs=127.1.32.126:45571
--webserver_port=43087
--webserver_interface=127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:10:48.718977 15481 log.cc:824] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:48.719002 15588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:46449 every 8 connection(s)
I20260620 23:10:48.721256 15481 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:10:48.721567 15481 tablet_bootstrap.cc:492] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Bootstrap complete.
I20260620 23:10:48.722640 15481 ts_tablet_manager.cc:1397] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
I20260620 23:10:48.723969 15481 raft_consensus.cc:348] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.724625 15481 raft_consensus.cc:729] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287afdcbfc8e447aac28c912edef9184, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.724875 15481 consensus_queue.cc:260] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } }
I20260620 23:10:48.725170 15481 ts_tablet_manager.cc:1428] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:48.730204 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.716237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:39309)
0620 23:10:48.718143 (+ 1906us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.718196 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.718785 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.719518 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.719521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.719918 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:48.720153 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.720159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.721725 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.721726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.723076 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.723079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.724017 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.724018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.724145 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.724670 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:48.724680 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:48.726408 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:48.728483 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.728489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.728490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.728766 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.729062 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.729062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.729063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.729178 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.729653 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:48.729658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:48.729788 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.729889 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.730024 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1852,"thread_start_us":119,"threads_started":1}
I20260620 23:10:48.730525 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.716337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:39309 (local address 127.1.32.126:45571)
0620 23:10:48.719025 (+ 2688us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.719033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.719066 (+ 33us) server_negotiation.cc:407] Connection header received
0620 23:10:48.719188 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.719195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.719266 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.719390 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:48.720271 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.721599 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.723173 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.723826 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.723928 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.726522 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:48.726526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:48.726526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:48.726535 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:48.728403 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.728854 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.728858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.728858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.728985 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.729253 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.729254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.729255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.729576 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:48.729670 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.729963 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.730085 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2628,"thread_start_us":75,"threads_started":1}
I20260620 23:10:48.731112 15589 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45571
I20260620 23:10:48.731189 15589 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:48.731343 15589 heartbeater.cc:511] Master 127.1.32.126:45571 requested a full tablet report, sending...
I20260620 23:10:48.731976 15401 ts_manager.cc:194] Registered new tserver with Master: 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449)
I20260620 23:10:48.732998 15401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:39309
I20260620 23:10:48.734557 15589 heartbeater.cc:503] Master 127.1.32.126:45571 was elected leader, sending a full tablet report...
I20260620 23:10:48.741529 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.733356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46449 (local address 127.0.0.1:51286)
0620 23:10:48.738315 (+ 4959us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.738340 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.738401 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.738562 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.738564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.738565 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.738607 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.738608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.739730 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.739730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.740477 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.740478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.741333 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.741334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.741388 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.741396 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.741448 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.741482 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4933,"thread_start_us":95,"threads_started":1}
I20260620 23:10:48.741564 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.733441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.1.32.65:46449)
0620 23:10:48.733569 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.733573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.738409 (+ 4836us) server_negotiation.cc:407] Connection header received
0620 23:10:48.738452 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.738455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.738468 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.738544 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.738663 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.739642 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.740554 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.741218 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.741311 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.741357 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.741476 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.741509 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260620 23:10:48.741868 15523 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4f0a9619d342568bb26ba48e941134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:48 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:51286
I20260620 23:10:48.742372 15598 tablet_replica.cc:330] stopping tablet replica
I20260620 23:10:48.742476 15598 raft_consensus.cc:2229] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:10:48.742554 15598 raft_consensus.cc:2258] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:48.742848 15598 ts_tablet_manager.cc:1905] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:10:48.744599 15598 ts_tablet_manager.cc:1918] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:10:48.744688 15598 log.cc:1196] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/8a4f0a9619d342568bb26ba48e941134
I20260620 23:10:48.744908 15598 ts_tablet_manager.cc:1939] T 8a4f0a9619d342568bb26ba48e941134 P 287afdcbfc8e447aac28c912edef9184: Deleting consensus metadata
I20260620 23:10:48.745422 15385 catalog_manager.cc:4841] TS 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449): tablet 8a4f0a9619d342568bb26ba48e941134 (table test-table [id=9ff8a8f410064a268e4b532a14612400]) successfully deleted
W20260620 23:10:48.796209 15593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:48.796380 15593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:48.796401 15593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:48.797434 15593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:48.797465 15593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:48.797490 15593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:10:48.798702 15593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:37509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=43087
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:48.798864 15593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:48.799124 15593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:48.803615 15604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.803596 15603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.803606 15606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:48.803862 15593 server_base.cc:1034] running on GCE node
I20260620 23:10:48.804075 15593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:48.804293 15593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:48.805454 15593 hybrid_clock.cc:648] HybridClock initialized: now 1781997048805434 us; error 30 us; skew 500 ppm
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:48.807781 15593 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:10:48.808203 15593 webserver.cc:460] Webserver started at http://127.1.32.66:43087/ using document root <none> and password file <none>
I20260620 23:10:48.808431 15593 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:48.808581 15593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:48.809916 15593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:10:48.810499 15613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.810665 15593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:10:48.810770 15593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "820d54f23d2a4dee9b7f4b17c1808ebb"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "c2077d519e2d709c4c99582006a8f472"
server_key_iv: "d8115d93e4da95e853406e857272b885"
server_key_version: "encryptionkey@0"
I20260620 23:10:48.811162 15593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:48.828766 15593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:48.830396 15593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:48.830544 15593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:48.830746 15593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:48.831044 15593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:48.831100 15593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.831132 15593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:48.831158 15593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.836726 15593 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:37509
I20260620 23:10:48.836789 15726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:37509 every 8 connection(s)
I20260620 23:10:48.837123 15593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:10:48.842089 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15593
I20260620 23:10:48.842360 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:34437
--local_ip_for_outbound_sockets=127.1.32.67
--tserver_master_addrs=127.1.32.126:45571
--webserver_port=40145
--webserver_interface=127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:48.846701 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.837864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:57849 (local address 127.1.32.126:45571)
0620 23:10:48.837920 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.837922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.838121 (+ 199us) server_negotiation.cc:407] Connection header received
0620 23:10:48.838572 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.838573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.838584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.838631 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:48.839619 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.840079 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.841075 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.841266 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.841313 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.843818 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:48.843820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:48.843820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:48.843829 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:48.845225 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.845702 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.845703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.845703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.845750 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.846026 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.846026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.846027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.846119 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:48.846167 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.846551 (+ 384us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.846618 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:10:48.847173 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.837862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:57849)
0620 23:10:48.837997 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.838028 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.838484 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.838692 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.838695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.839337 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:48.839547 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.839551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.840168 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.840169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.840979 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.840981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.841346 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.841346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.841440 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.841936 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:48.841943 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:48.843730 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:48.845305 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.845318 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.845320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.845630 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.845815 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.845816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.845817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.845961 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.846201 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:48.846207 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:48.846386 (+ 179us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.846534 (+ 148us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.846705 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260620 23:10:48.847699 15727 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45571
I20260620 23:10:48.847805 15727 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:48.848012 15727 heartbeater.cc:511] Master 127.1.32.126:45571 requested a full tablet report, sending...
I20260620 23:10:48.848407 15401 ts_manager.cc:194] Registered new tserver with Master: 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509)
I20260620 23:10:48.848909 15401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:57849
W20260620 23:10:48.920804 15730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:48.921013 15730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:48.921032 15730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:48.922015 15730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:48.922045 15730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:48.922067 15730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:10:48.923341 15730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:34437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=40145
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:48.923516 15730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:48.923668 15730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:48.928644 15738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.928640 15735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:48.928644 15736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:48.929071 15730 server_base.cc:1034] running on GCE node
I20260620 23:10:48.929241 15730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:48.929463 15730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:48.930613 15730 hybrid_clock.cc:648] HybridClock initialized: now 1781997048930601 us; error 27 us; skew 500 ppm
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:48.932997 15730 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:10:48.933401 15730 webserver.cc:460] Webserver started at http://127.1.32.67:40145/ using document root <none> and password file <none>
I20260620 23:10:48.933588 15730 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:48.933633 15730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:48.934872 15730 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:48.935474 15745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.935652 15730 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.935730 15730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "4fc3e3c1b42947aa919116e60bf330cc"
format_stamp: "Formatted at 2026-06-20 23:10:48 on dist-test-slave-fzjm"
server_key: "1592ae6e7dba1111e76db0620625ce87"
server_key_iv: "8e49e072a5761037b31364604df52244"
server_key_version: "encryptionkey@0"
I20260620 23:10:48.936043 15730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:48.954084 15730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:48.956470 15730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:48.956630 15730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:48.956856 15730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:48.957145 15730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:48.957180 15730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.957203 15730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:48.957252 15730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.963660 15730 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:34437
I20260620 23:10:48.963711 15858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:34437 every 8 connection(s)
I20260620 23:10:48.964071 15730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:10:48.967484 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15730
Jun 20 23:10:48 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997048, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:10:48.973374 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.964726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:51731 (local address 127.1.32.126:45571)
0620 23:10:48.964797 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.964800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.965069 (+ 269us) server_negotiation.cc:407] Connection header received
0620 23:10:48.965603 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.965604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.965613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.965662 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:48.966508 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.966976 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.968053 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.968263 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.968347 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.970452 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:48.970453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:48.970453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:48.970459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:48.971885 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.972371 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.972372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.972373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.972432 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.972722 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.972722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.972722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.972817 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:48.972868 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.973178 (+ 310us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.973254 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 23:10:48.973503 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.964740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:51731)
0620 23:10:48.964936 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.964968 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.965517 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.965712 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.965714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.966247 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:48.966427 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.966432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.967042 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.967043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.967965 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.967968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.968351 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.968352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.968477 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.968995 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:48.969003 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:48.970331 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:48.971953 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.971959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.971961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.972263 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.972507 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:48.972508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:48.972508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:48.972647 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:48.972883 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:48.972889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:48.973036 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.973149 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.973305 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260620 23:10:48.973954 15859 heartbeater.cc:347] Connected to a master server at 127.1.32.126:45571
I20260620 23:10:48.974058 15859 heartbeater.cc:464] Registering TS with master...
I20260620 23:10:48.974264 15859 heartbeater.cc:511] Master 127.1.32.126:45571 requested a full tablet report, sending...
I20260620 23:10:48.974664 15401 ts_manager.cc:194] Registered new tserver with Master: 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
I20260620 23:10:48.974764 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.967913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.1.32.126:45571)
0620 23:10:48.968025 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.968027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.968035 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:10:48.968058 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.968059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.968069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.968129 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:48.968634 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.969081 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.969849 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.970071 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.970166 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.971563 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:48.971564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:48.971565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:48.971570 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:48.973201 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.973886 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.973887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.973888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.973954 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:48.974191 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:48.974192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:48.974192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:48.974316 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:48.974489 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.974562 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.974651 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
I20260620 23:10:48.975144 15401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:51731
I20260620 23:10:48.975540 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:10:48.977946 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.976000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.1.32.126:45571)
0620 23:10:48.976049 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.976051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.976059 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:10:48.976087 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.976087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.976121 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.976177 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:10:48.976260 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.976665 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.977356 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.977526 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.977575 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.977716 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:10:48.977791 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:10:48.977845 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.977889 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.977904 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 23:10:48.978226 15400 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:10:48.979915 15400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:10:48.982059 15523 tablet_service.cc:1467] Processing CreateTablet for tablet a1f686d032854829a02af522f902da22 (DEFAULT_TABLE table=test-table [id=ef27c87812324084a961676aba7b8467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:48.982263 15523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f686d032854829a02af522f902da22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:48.983662 15481 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Bootstrap starting.
I20260620 23:10:48.983990 15481 tablet_bootstrap.cc:654] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:48.984563 15481 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: No bootstrap required, opened a new log
I20260620 23:10:48.984633 15481 ts_tablet_manager.cc:1397] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:48.984740 15481 raft_consensus.cc:348] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:48.984810 15481 raft_consensus.cc:374] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:48.984858 15481 raft_consensus.cc:729] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287afdcbfc8e447aac28c912edef9184, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.984900 15481 consensus_queue.cc:260] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:48.984987 15481 ts_tablet_manager.cc:1428] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:48.985349 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.981809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:34437 (local address 127.0.0.1:59198)
0620 23:10:48.981951 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.981961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.982006 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.982402 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.982403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.982405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.982444 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.982445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.983376 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.983377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.984179 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.984180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.985027 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.985028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.985116 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.985119 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.985170 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.985206 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260620 23:10:48.985646 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.982122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59198 (local address 127.1.32.67:34437)
0620 23:10:48.982253 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.982257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.982265 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:10:48.982301 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.982302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.982313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.982405 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.982497 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.983278 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.984274 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.984877 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.984940 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.984995 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.985539 (+ 544us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.985576 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260620 23:10:48.986166 15793 tablet_service.cc:1467] Processing CreateTablet for tablet a1f686d032854829a02af522f902da22 (DEFAULT_TABLE table=test-table [id=ef27c87812324084a961676aba7b8467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:48.986406 15793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f686d032854829a02af522f902da22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:48.987088 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.981767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:37509 (local address 127.0.0.1:44570)
0620 23:10:48.981943 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:48.981955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:48.981995 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:48.983929 (+ 1934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:48.983930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:48.983931 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:48.983959 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.983960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.985147 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.985148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.986047 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:48.986049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.986853 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.986854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:48.986933 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.986937 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:10:48.986992 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 23:10:48.987030 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260620 23:10:48.987115 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:48.981870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.1.32.66:37509)
0620 23:10:48.983748 (+ 1878us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:48.983752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:48.983763 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:10:48.983819 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:48.983821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:48.983838 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:48.983926 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:48.984013 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.984970 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.986134 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:48.986751 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:48.986853 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:48.986923 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:48.987005 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 23:10:48.987043 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1844,"thread_start_us":98,"threads_started":1}
I20260620 23:10:48.987747 15661 tablet_service.cc:1467] Processing CreateTablet for tablet a1f686d032854829a02af522f902da22 (DEFAULT_TABLE table=test-table [id=ef27c87812324084a961676aba7b8467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:10:48.988029 15661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f686d032854829a02af522f902da22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:10:48.988962 15869 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Bootstrap starting.
I20260620 23:10:48.989537 15869 tablet_bootstrap.cc:654] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:48.989976 15869 log.cc:824] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:48.990818 15869 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: No bootstrap required, opened a new log
I20260620 23:10:48.990873 15869 ts_tablet_manager.cc:1397] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:10:48.991029 15871 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Bootstrap starting.
I20260620 23:10:48.991544 15871 tablet_bootstrap.cc:654] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Neither blocks nor log segments found. Creating new log.
I20260620 23:10:48.992026 15871 log.cc:824] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:48.992448 15869 raft_consensus.cc:348] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:48.992610 15869 raft_consensus.cc:374] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:48.992660 15869 raft_consensus.cc:729] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc3e3c1b42947aa919116e60bf330cc, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.992726 15871 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: No bootstrap required, opened a new log
I20260620 23:10:48.992795 15871 ts_tablet_manager.cc:1397] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:10:48.992795 15869 consensus_queue.cc:260] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:48.993103 15869 ts_tablet_manager.cc:1428] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 23:10:48.993105 15859 heartbeater.cc:503] Master 127.1.32.126:45571 was elected leader, sending a full tablet report...
I20260620 23:10:48.994565 15871 raft_consensus.cc:348] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:48.994704 15871 raft_consensus.cc:374] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:10:48.994725 15871 raft_consensus.cc:729] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820d54f23d2a4dee9b7f4b17c1808ebb, State: Initialized, Role: FOLLOWER
I20260620 23:10:48.994825 15871 consensus_queue.cc:260] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:48.995096 15871 ts_tablet_manager.cc:1428] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:10:48.995213 15727 heartbeater.cc:503] Master 127.1.32.126:45571 was elected leader, sending a full tablet report...
I20260620 23:10:49.008452 15873 raft_consensus.cc:482] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:49.008538 15873 raft_consensus.cc:504] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:49.008917 15873 leader_election.cc:288] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509), 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449)
I20260620 23:10:49.011615 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.009062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:37509 (local address 127.1.32.67:49353)
0620 23:10:49.009110 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.009121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:49.009160 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:49.009374 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:49.009375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:49.009376 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:49.009441 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:49.009442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.010047 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.010048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:49.010721 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:49.010722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.011358 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.011359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:49.011439 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:49.011442 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:10:49.011510 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:10:49.011550 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 23:10:49.011613 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.009194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:49353 (local address 127.1.32.66:37509)
0620 23:10:49.009243 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:49.009245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:49.009251 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:10:49.009276 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:49.009277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:49.009288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:49.009369 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:49.009507 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.009959 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.010841 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.011257 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.011356 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:49.011366 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:49.011518 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:10:49.011550 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:10:49.011843 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.009136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46449 (local address 127.1.32.67:50735)
0620 23:10:49.009343 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.009351 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:49.009424 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:49.009569 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:49.009569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:49.009571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:10:49.009636 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:49.009637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.010227 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.010228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:49.011032 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:49.011034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.011555 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.011556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:49.011622 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:49.011631 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 23:10:49.011698 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 23:10:49.011742 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
I20260620 23:10:49.011933 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.009231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:50735 (local address 127.1.32.65:46449)
0620 23:10:49.009305 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:49.009307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:49.009412 (+ 105us) server_negotiation.cc:407] Connection header received
0620 23:10:49.009477 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:49.009478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:49.009490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:49.009554 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:10:49.009705 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.010146 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.011113 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.011476 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.011555 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:49.011563 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:49.011821 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 23:10:49.011862 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 23:10:49.011919 15681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f686d032854829a02af522f902da22" candidate_uuid: "4fc3e3c1b42947aa919116e60bf330cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" is_pre_election: true
I20260620 23:10:49.012051 15681 raft_consensus.cc:2454] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fc3e3c1b42947aa919116e60bf330cc in term 0.
I20260620 23:10:49.012149 15543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f686d032854829a02af522f902da22" candidate_uuid: "4fc3e3c1b42947aa919116e60bf330cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "287afdcbfc8e447aac28c912edef9184" is_pre_election: true
I20260620 23:10:49.012283 15747 leader_election.cc:302] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fc3e3c1b42947aa919116e60bf330cc, 820d54f23d2a4dee9b7f4b17c1808ebb; no voters:
I20260620 23:10:49.012319 15543 raft_consensus.cc:2454] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fc3e3c1b42947aa919116e60bf330cc in term 0.
I20260620 23:10:49.012393 15873 raft_consensus.cc:2781] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:10:49.012528 15873 raft_consensus.cc:482] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:10:49.012553 15873 raft_consensus.cc:3037] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:49.013319 15873 raft_consensus.cc:504] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:49.013463 15873 leader_election.cc:288] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [CANDIDATE]: Term 1 election: Requested vote from peers 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509), 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449)
I20260620 23:10:49.013661 15543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f686d032854829a02af522f902da22" candidate_uuid: "4fc3e3c1b42947aa919116e60bf330cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "287afdcbfc8e447aac28c912edef9184"
I20260620 23:10:49.013697 15681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f686d032854829a02af522f902da22" candidate_uuid: "4fc3e3c1b42947aa919116e60bf330cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb"
I20260620 23:10:49.013764 15543 raft_consensus.cc:3037] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:49.013764 15681 raft_consensus.cc:3037] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:10:49.014492 15681 raft_consensus.cc:2454] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fc3e3c1b42947aa919116e60bf330cc in term 1.
I20260620 23:10:49.014662 15747 leader_election.cc:302] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fc3e3c1b42947aa919116e60bf330cc, 820d54f23d2a4dee9b7f4b17c1808ebb; no voters:
I20260620 23:10:49.014739 15873 raft_consensus.cc:2781] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:10:49.014942 15873 raft_consensus.cc:686] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 LEADER]: Becoming Leader. State: Replica: 4fc3e3c1b42947aa919116e60bf330cc, State: Running, Role: LEADER
I20260620 23:10:49.015120 15873 consensus_queue.cc:237] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:49.015169 15543 raft_consensus.cc:2454] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fc3e3c1b42947aa919116e60bf330cc in term 1.
I20260620 23:10:49.015965 15400 catalog_manager.cc:5495] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67). New cstate: current_term: 1 leader_uuid: "4fc3e3c1b42947aa919116e60bf330cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } health_report { overall_health: UNKNOWN } } }
I20260620 23:10:49.024713 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.022216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59208 (local address 127.1.32.67:34437)
0620 23:10:49.022269 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:49.022272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:49.022282 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:10:49.022345 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:49.022346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:49.022365 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:49.022463 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:10:49.022583 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.023110 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.023878 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:49.024067 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:49.024160 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:49.024360 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:10:49.024470 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:10:49.024572 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:49.024594 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 23:10:49.024613 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 23:10:49.026274 15681 raft_consensus.cc:1261] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 FOLLOWER]: Refusing update from remote peer 4fc3e3c1b42947aa919116e60bf330cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:49.026269 15543 raft_consensus.cc:1261] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Refusing update from remote peer 4fc3e3c1b42947aa919116e60bf330cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:10:49.026621 15873 consensus_queue.cc:1035] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:49.026778 15876 consensus_queue.cc:1035] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:10:49.028831 15879 mvcc.cc:205] Tried to move back new op lower bound from 7299059912809279488 to 7299059912766652416. Current Snapshot: MvccSnapshot[applied={T|T < 7299059912809279488}]
I20260620 23:10:49.030326 15882 mvcc.cc:205] Tried to move back new op lower bound from 7299059912809279488 to 7299059912766652416. Current Snapshot: MvccSnapshot[applied={T|T < 7299059912809279488 or (T in {7299059912809279488})}]
I20260620 23:10:49.031243 15400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:10:49.031355 15400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:10:49.032117 15400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Sending DeleteTablet for 3 replicas of tablet a1f686d032854829a02af522f902da22
I20260620 23:10:49.032438 15793 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f686d032854829a02af522f902da22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:49 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:59198
I20260620 23:10:49.032446 15523 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f686d032854829a02af522f902da22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:49 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:51286
I20260620 23:10:49.032557 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 15439
I20260620 23:10:49.032580 15661 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f686d032854829a02af522f902da22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:10:49 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:44570
I20260620 23:10:49.032733 15598 tablet_replica.cc:330] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: stopping tablet replica
I20260620 23:10:49.032783 15598 raft_consensus.cc:2229] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:10:49.032829 15598 raft_consensus.cc:2258] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:49.033303 15889 tablet_replica.cc:330] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: stopping tablet replica
I20260620 23:10:49.033320 15888 tablet_replica.cc:330] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: stopping tablet replica
I20260620 23:10:49.033401 15888 raft_consensus.cc:2229] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:10:49.033430 15889 raft_consensus.cc:2229] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:10:49.033514 15888 raft_consensus.cc:2258] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:49.033628 15889 raft_consensus.cc:2258] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:10:49.033962 15889 ts_tablet_manager.cc:1905] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:10:49.035305 15889 ts_tablet_manager.cc:1918] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:10:49.035403 15889 log.cc:1196] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/a1f686d032854829a02af522f902da22
I20260620 23:10:49.035634 15889 ts_tablet_manager.cc:1939] T a1f686d032854829a02af522f902da22 P 820d54f23d2a4dee9b7f4b17c1808ebb: Deleting consensus metadata
I20260620 23:10:49.036031 15386 catalog_manager.cc:4841] TS 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509): tablet a1f686d032854829a02af522f902da22 (table test-table [id=ef27c87812324084a961676aba7b8467]) successfully deleted
W20260620 23:10:49.037242 15385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:46449 (error 108)
I20260620 23:10:49.037638 15888 ts_tablet_manager.cc:1905] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:10:49.037707 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.037584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:46449 (local address 127.0.0.1:51290)
0620 23:10:49.037633 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.037663 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:10:49.037804 15385 catalog_manager.cc:4584] TS 287afdcbfc8e447aac28c912edef9184 (127.1.32.65:46449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1f686d032854829a02af522f902da22: Network error: Client connection negotiation failed: client connection to 127.1.32.65:46449: connect: Connection refused (error 111)
I20260620 23:10:49.039057 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 15593
I20260620 23:10:49.039741 15888 ts_tablet_manager.cc:1918] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:10:49.039811 15888 log.cc:1196] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/a1f686d032854829a02af522f902da22
I20260620 23:10:49.040103 15888 ts_tablet_manager.cc:1939] T a1f686d032854829a02af522f902da22 P 4fc3e3c1b42947aa919116e60bf330cc: Deleting consensus metadata
I20260620 23:10:49.042055 15388 catalog_manager.cc:4841] TS 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437): tablet a1f686d032854829a02af522f902da22 (table test-table [id=ef27c87812324084a961676aba7b8467]) successfully deleted
I20260620 23:10:49.045850 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 15730
I20260620 23:10:49.050078 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 15368
I20260620 23:10:49.054976 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45571
--webserver_interface=127.1.32.126
--webserver_port=41869
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:45571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:49.132161 15891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:49.132345 15891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:49.132368 15891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:49.133450 15891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:10:49.133497 15891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:49.133513 15891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:49.133548 15891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:10:49.133563 15891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:10:49.135010 15891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:45571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:45571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=41869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:49.135178 15891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:49.135334 15891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:49.139736 15897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:49.139725 15896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:49.139845 15891 server_base.cc:1034] running on GCE node
W20260620 23:10:49.139715 15899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:49.140106 15891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:49.140373 15891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:49.141530 15891 hybrid_clock.cc:648] HybridClock initialized: now 1781997049141528 us; error 36 us; skew 500 ppm
Jun 20 23:10:49 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997049, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:49.143990 15891 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:10:49.144418 15891 webserver.cc:460] Webserver started at http://127.1.32.126:41869/ using document root <none> and password file <none>
I20260620 23:10:49.144642 15891 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:49.144683 15891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:49.145893 15891 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:10:49.146575 15906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.146757 15891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:10:49.146842 15891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "8656af006d9f489da448d56f56926c59"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "3be30ba469ad15044e51a76743fce8f9"
server_key_iv: "21455c8ea2452e2dfb9174ca893e5bc1"
server_key_version: "encryptionkey@0"
I20260620 23:10:49.147210 15891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:49.180626 15891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:49.182367 15891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:49.182509 15891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:49.186575 15891 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:45571
I20260620 23:10:49.186630 15958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:45571 every 8 connection(s)
I20260620 23:10:49.186982 15891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:10:49.188562 15959 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:10:49.189102 15959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Bootstrap starting.
I20260620 23:10:49.189330 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15891
I20260620 23:10:49.189618 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:46449
--local_ip_for_outbound_sockets=127.1.32.65
--tserver_master_addrs=127.1.32.126:45571
--webserver_port=38535
--webserver_interface=127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:10:49.197793 15959 log.cc:824] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:49.199887 15959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:10:49.200168 15959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59: Bootstrap complete.
I20260620 23:10:49.202317 15959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:49.202522 15959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8656af006d9f489da448d56f56926c59, State: Initialized, Role: FOLLOWER
I20260620 23:10:49.202716 15959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:49.202772 15959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:10:49.202827 15959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:10:49.202870 15959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:10:49.203843 15959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:49.203926 15959 leader_election.cc:302] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8656af006d9f489da448d56f56926c59; no voters:
I20260620 23:10:49.204121 15959 leader_election.cc:288] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 23:10:49.204193 15963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 23:10:49.204335 15963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [term 3 LEADER]: Becoming Leader. State: Replica: 8656af006d9f489da448d56f56926c59, State: Running, Role: LEADER
I20260620 23:10:49.204397 15959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:10:49.204432 15963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } }
I20260620 23:10:49.204682 15963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8656af006d9f489da448d56f56926c59. Latest consensus state: current_term: 3 leader_uuid: "8656af006d9f489da448d56f56926c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } } }
I20260620 23:10:49.204742 15963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:49.204871 15964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8656af006d9f489da448d56f56926c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8656af006d9f489da448d56f56926c59" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 45571 } } }
I20260620 23:10:49.204921 15964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8656af006d9f489da448d56f56926c59 [sys.catalog]: This master's current role is: LEADER
I20260620 23:10:49.205004 15968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:10:49.205685 15968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:10:49.205780 15968 catalog_manager.cc:1259] Loaded cluster ID: 67fbe18465be41d4b3001d61fb5137b1
I20260620 23:10:49.205811 15968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 23:10:49.206022 15968 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbc019a19ad google::LogMessage::Fail() at ??:0
@ 0x7fbc019a488a google::LogMessage::SendToLog() at ??:0
@ 0x7fbc019a14de google::LogMessage::Flush() at ??:0
@ 0x7fbc019a2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbc098c20f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbc098cb8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbc0386f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbc03864954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbc046c86db start_thread at ??:0
@ 0x7fbc0049f61f clone at ??:0
W20260620 23:10:49.283288 15961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:49.283612 15961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:49.283645 15961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:49.284840 15961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:49.284904 15961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:49.284948 15961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:10:49.286334 15961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:46449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=38535
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:49.286571 15961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:49.286824 15961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:10:49.291553 15961 server_base.cc:1034] running on GCE node
W20260620 23:10:49.291522 15998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:49.291538 15996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:49.291759 15995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:49.292017 15961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:49.292294 15961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:49.293468 15961 hybrid_clock.cc:648] HybridClock initialized: now 1781997049293445 us; error 39 us; skew 500 ppm
Jun 20 23:10:49 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997049, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:49.296519 15961 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:10:49.296998 15961 webserver.cc:460] Webserver started at http://127.1.32.65:38535/ using document root <none> and password file <none>
I20260620 23:10:49.297225 15961 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:49.297287 15961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:49.299011 15961 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:10:49.299731 16005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.299908 15961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.299998 15961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "287afdcbfc8e447aac28c912edef9184"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "940276e3ac0e400b78cf2debdafed32d"
server_key_iv: "65b45c138c4aa43c4ce4458a3a2cfb9d"
server_key_version: "encryptionkey@0"
I20260620 23:10:49.300287 15961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:49.307546 15961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:49.310159 15961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:49.310321 15961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:49.310590 15961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:49.311106 16012 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:10:49.312436 15961 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:10:49.312508 15961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 23:10:49.312538 15961 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:10:49.313396 15961 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:10:49.313441 15961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:10:49.313508 16012 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Bootstrap starting.
I20260620 23:10:49.320734 15961 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:46449
I20260620 23:10:49.321106 16119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:46449 every 8 connection(s)
I20260620 23:10:49.321262 15961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:10:49.324226 16012 log.cc:824] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Log is configured to *not* fsync() on all Append() calls
I20260620 23:10:49.326146 16012 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:10:49.326437 16012 tablet_bootstrap.cc:492] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Bootstrap complete.
I20260620 23:10:49.327390 16012 ts_tablet_manager.cc:1397] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.013s
I20260620 23:10:49.327819 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 15961
I20260620 23:10:49.328094 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:37509
--local_ip_for_outbound_sockets=127.1.32.66
--tserver_master_addrs=127.1.32.126:45571
--webserver_port=43087
--webserver_interface=127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:10:49.329059 16012 raft_consensus.cc:348] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:49.329422 16012 raft_consensus.cc:729] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287afdcbfc8e447aac28c912edef9184, State: Initialized, Role: FOLLOWER
I20260620 23:10:49.329588 16012 consensus_queue.cc:260] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:49.329857 16012 ts_tablet_manager.cc:1428] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260620 23:10:49.442677 16124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:49.442956 16124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:49.442996 16124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:49.444602 16124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:49.444679 16124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:49.444716 16124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:10:49.447482 16124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:37509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=43087
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:49.447713 16124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:49.447924 16124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:49.452126 16122 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:10:49.321916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:58315)
0620 23:10:49.322103 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.322145 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:49.322813 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:49.451818 (+129005us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
W20260620 23:10:49.452404 16120 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:45571 (0 consecutive failures): Network error: Failed to ping master at 127.1.32.126:45571: Client connection negotiation failed: client connection to 127.1.32.126:45571: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 23:10:49.452582 16131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:49.452705 16133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:49.452828 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.452646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:56787)
0620 23:10:49.452764 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.452805 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260620 23:10:49.453171 16130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:49.453233 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.453145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:54431)
0620 23:10:49.453187 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.453207 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 23:10:49.453292 16124 server_base.cc:1034] running on GCE node
W20260620 23:10:49.453341 16120 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:10:49.453490 16124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:49.453732 16124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:49.454947 16124 hybrid_clock.cc:648] HybridClock initialized: now 1781997049454930 us; error 24 us; skew 500 ppm
Jun 20 23:10:49 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997049, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:49.457257 16124 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:10:49.457674 16124 webserver.cc:460] Webserver started at http://127.1.32.66:43087/ using document root <none> and password file <none>
I20260620 23:10:49.457897 16124 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:49.457962 16124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:49.459288 16124 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:10:49.459913 16140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.460074 16124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:10:49.460165 16124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "820d54f23d2a4dee9b7f4b17c1808ebb"
format_stamp: "Formatted at 2026-06-20 23:10:47 on dist-test-slave-fzjm"
server_key: "c2077d519e2d709c4c99582006a8f472"
server_key_iv: "d8115d93e4da95e853406e857272b885"
server_key_version: "encryptionkey@0"
I20260620 23:10:49.460511 16124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:49.474380 16124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:49.476243 16124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:49.476403 16124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:49.476619 16124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:49.476895 16124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:49.476949 16124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.477003 16124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:49.477033 16124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.482636 16124 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:37509
I20260620 23:10:49.482707 16253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:37509 every 8 connection(s)
I20260620 23:10:49.483095 16124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:10:49.484051 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 16124
I20260620 23:10:49.484112 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.483747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:60243)
0620 23:10:49.483906 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.483968 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260620 23:10:49.484302 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:34437
--local_ip_for_outbound_sockets=127.1.32.67
--tserver_master_addrs=127.1.32.126:45571
--webserver_port=40145
--webserver_interface=127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:10:49.484525 16254 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:45571 (0 consecutive failures): Network error: Failed to ping master at 127.1.32.126:45571: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
I20260620 23:10:49.484798 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.484704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:41535)
0620 23:10:49.484753 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.484778 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:10:49.485071 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.484985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:51341)
0620 23:10:49.485032 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.485053 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:10:49.485196 16254 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:10:49.561010 16257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:10:49.561179 16257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:10:49.561201 16257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:10:49.562274 16257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:10:49.562314 16257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:10:49.562340 16257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:10:49.563880 16257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:34437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=40145
--tserver_master_addrs=127.1.32.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:10:49.564145 16257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:10:49.564378 16257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:10:49.569209 16263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:49.569203 16262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:10:49.569207 16265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:10:49.569450 16257 server_base.cc:1034] running on GCE node
I20260620 23:10:49.569645 16257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:10:49.569883 16257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:10:49.571002 16257 hybrid_clock.cc:648] HybridClock initialized: now 1781997049570986 us; error 31 us; skew 500 ppm
Jun 20 23:10:49 dist-test-slave-fzjm krb5kdc[14824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997049, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:10:49.573275 16257 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:10:49.573649 16257 webserver.cc:460] Webserver started at http://127.1.32.67:40145/ using document root <none> and password file <none>
I20260620 23:10:49.573851 16257 fs_manager.cc:359] Metadata directory not provided
I20260620 23:10:49.573892 16257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:10:49.575153 16257 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:10:49.575651 16272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.575789 16257 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.575896 16257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "4fc3e3c1b42947aa919116e60bf330cc"
format_stamp: "Formatted at 2026-06-20 23:10:48 on dist-test-slave-fzjm"
server_key: "1592ae6e7dba1111e76db0620625ce87"
server_key_iv: "8e49e072a5761037b31364604df52244"
server_key_version: "encryptionkey@0"
I20260620 23:10:49.576195 16257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:10:49.596218 16257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:10:49.598035 16257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:10:49.598196 16257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:10:49.598435 16257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:10:49.598732 16257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:10:49.598783 16257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.598830 16257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:10:49.598860 16257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:10:49.604980 16257 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:34437
I20260620 23:10:49.605021 16385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:34437 every 8 connection(s)
I20260620 23:10:49.605346 16257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:10:49.606328 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.605963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:59845)
0620 23:10:49.606108 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.606177 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
W20260620 23:10:49.606737 16386 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:45571 (0 consecutive failures): Network error: Failed to ping master at 127.1.32.126:45571: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
I20260620 23:10:49.607066 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.606972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:41975)
0620 23:10:49.607008 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.607034 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260620 23:10:49.607338 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:49.607263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:35153)
0620 23:10:49.607304 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:49.607324 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 23:10:49.607473 16386 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:10:49.609177 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 16257
I20260620 23:10:50.453843 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:50.453629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:39369)
0620 23:10:50.453742 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:50.453769 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260620 23:10:50.485694 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:50.485478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:38439)
0620 23:10:50.485602 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:50.485651 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260620 23:10:50.605746 16392 raft_consensus.cc:482] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:50.605895 16392 raft_consensus.cc:504] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:50.606210 16392 leader_election.cc:288] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509), 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
I20260620 23:10:50.608011 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:50.607796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:50543)
0620 23:10:50.607911 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:50.607957 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
Jun 20 23:10:50 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997049, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.66@KRBTEST.COM
Jun 20 23:10:50 dist-test-slave-fzjm krb5kdc[14824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997049, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.67@KRBTEST.COM
I20260620 23:10:50.620929 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:50.606486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:40723 (local address 127.1.32.66:37509)
0620 23:10:50.606636 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:50.606644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:50.606667 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:10:50.607092 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:50.607094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:50.607129 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:50.607264 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:50.608018 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.609091 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.610074 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.610461 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.610574 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:50.617204 (+ 6630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:50.617209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:50.617209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:50.617221 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:50.619220 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:50.619709 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:50.619713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:50.619713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:50.619831 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:50.620126 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:50.620127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:50.620127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:50.620352 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:50.620455 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:50.620699 (+ 244us) server_negotiation.cc:299] Negotiation successful
0620 23:10:50.620831 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":47,"threads_started":1}
I20260620 23:10:50.620915 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:50.606374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:34437 (local address 127.1.32.65:37691)
0620 23:10:50.606502 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:50.606510 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:50.606546 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:50.607278 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:50.607281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:50.607710 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:50.607906 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:50.607913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.609178 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.609179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:50.609977 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:50.609979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.610552 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.610553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:50.610667 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:50.615666 (+ 4999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:50.615671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:50.617508 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:50.619499 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:50.619499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:50.619500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:50.619731 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:50.619984 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:50.619985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:50.619985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:50.620034 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:50.620408 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:50.620415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:50.620533 (+ 118us) client_negotiation.cc:769] Sending connection context
0620 23:10:50.620635 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 23:10:50.620816 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":36,"thread_start_us":45,"threads_started":1}
I20260620 23:10:50.620946 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:50.606470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:37691 (local address 127.1.32.67:34437)
0620 23:10:50.606627 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 23:10:50.606636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:10:50.606653 (+ 17us) server_negotiation.cc:407] Connection header received
0620 23:10:50.607063 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:10:50.607065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:10:50.607104 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:10:50.607210 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:10:50.608028 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.609082 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.610063 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.610460 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.610575 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:50.617611 (+ 7036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:10:50.617615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:10:50.617615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:10:50.617623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:10:50.619410 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:50.619807 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:50.619809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:50.619809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:50.619917 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:10:50.620094 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:10:50.620095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:10:50.620095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:10:50.620333 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:10:50.620391 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:10:50.620691 (+ 300us) server_negotiation.cc:299] Negotiation successful
0620 23:10:50.620842 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
I20260620 23:10:50.621362 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:50.606374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:37509 (local address 127.1.32.65:40723)
0620 23:10:50.606445 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:50.606455 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:10:50.606504 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:10:50.607281 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:10:50.607282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:10:50.607712 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:10:50.607907 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:50.607913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.609200 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.609206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:50.609962 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:10:50.609964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:10:50.610553 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:10:50.610554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:10:50.610666 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:10:50.615632 (+ 4966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:10:50.615641 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:10:50.617097 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:10:50.619289 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:50.619295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:50.619298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:50.619632 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:50.619897 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:10:50.619898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:10:50.619898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:10:50.620027 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:10:50.620451 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:10:50.620452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:10:50.620533 (+ 81us) client_negotiation.cc:769] Sending connection context
0620 23:10:50.620635 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 23:10:50.620891 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":200}
W20260620 23:10:50.621752 16009 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437): Not found: Tablet not found: a1f686d032854829a02af522f902da22
W20260620 23:10:50.622018 16007 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509): Not found: Tablet not found: a1f686d032854829a02af522f902da22
I20260620 23:10:50.622068 16007 leader_election.cc:302] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 287afdcbfc8e447aac28c912edef9184; no voters: 4fc3e3c1b42947aa919116e60bf330cc, 820d54f23d2a4dee9b7f4b17c1808ebb
I20260620 23:10:50.622181 16392 raft_consensus.cc:2726] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:10:51.454474 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:51.454262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:34693)
0620 23:10:51.454390 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:51.454428 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260620 23:10:51.486270 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:51.486069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:37803)
0620 23:10:51.486201 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:51.486234 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260620 23:10:51.608580 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:51.608387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:54195)
0620 23:10:51.608506 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:51.608542 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260620 23:10:52.174314 16400 raft_consensus.cc:482] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:52.174431 16400 raft_consensus.cc:504] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:52.174597 16400 leader_election.cc:288] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509), 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
W20260620 23:10:52.175022 16007 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509): Not found: Tablet not found: a1f686d032854829a02af522f902da22
W20260620 23:10:52.175096 16009 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437): Not found: Tablet not found: a1f686d032854829a02af522f902da22
I20260620 23:10:52.175243 16007 leader_election.cc:302] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 287afdcbfc8e447aac28c912edef9184; no voters: 4fc3e3c1b42947aa919116e60bf330cc, 820d54f23d2a4dee9b7f4b17c1808ebb
I20260620 23:10:52.175340 16400 raft_consensus.cc:2726] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:10:52.455240 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:52.455027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:54851)
0620 23:10:52.455155 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:52.455192 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260620 23:10:52.486963 16402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:52.486704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:40547)
0620 23:10:52.486836 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:52.486872 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260620 23:10:52.609318 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:52.609084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:32811)
0620 23:10:52.609225 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:52.609264 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260620 23:10:53.455899 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:53.455685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:34383)
0620 23:10:53.455820 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:53.455853 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260620 23:10:53.487569 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:53.487352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:48439)
0620 23:10:53.487483 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:53.487520 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260620 23:10:53.609979 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:53.609763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:44807)
0620 23:10:53.609903 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:53.609939 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":57,"threads_started":1}
I20260620 23:10:54.407222 16407 raft_consensus.cc:482] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:54.407317 16407 raft_consensus.cc:504] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:54.407505 16407 leader_election.cc:288] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509), 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
W20260620 23:10:54.407913 16007 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509): Not found: Tablet not found: a1f686d032854829a02af522f902da22
W20260620 23:10:54.407984 16009 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437): Not found: Tablet not found: a1f686d032854829a02af522f902da22
I20260620 23:10:54.408067 16009 leader_election.cc:302] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 287afdcbfc8e447aac28c912edef9184; no voters: 4fc3e3c1b42947aa919116e60bf330cc, 820d54f23d2a4dee9b7f4b17c1808ebb
I20260620 23:10:54.408162 16407 raft_consensus.cc:2726] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:10:54.456666 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:54.456414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:42353)
0620 23:10:54.456574 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:54.456610 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
I20260620 23:10:54.488260 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:54.488027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:43383)
0620 23:10:54.488150 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:54.488213 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260620 23:10:54.610675 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:54.610453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:58337)
0620 23:10:54.610594 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:54.610632 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260620 23:10:55.457463 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:55.457217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:52495)
0620 23:10:55.457382 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:55.457416 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260620 23:10:55.489030 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:55.488846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:32885)
0620 23:10:55.488954 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:55.488990 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260620 23:10:55.611419 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:55.611159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:60865)
0620 23:10:55.611339 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:55.611370 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260620 23:10:56.080402 16414 raft_consensus.cc:482] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:10:56.080529 16414 raft_consensus.cc:504] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:10:56.080735 16414 leader_election.cc:288] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509), 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
W20260620 23:10:56.081131 16007 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509): Not found: Tablet not found: a1f686d032854829a02af522f902da22
W20260620 23:10:56.081235 16009 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437): Not found: Tablet not found: a1f686d032854829a02af522f902da22
I20260620 23:10:56.081315 16009 leader_election.cc:302] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 287afdcbfc8e447aac28c912edef9184; no voters: 4fc3e3c1b42947aa919116e60bf330cc, 820d54f23d2a4dee9b7f4b17c1808ebb
I20260620 23:10:56.081403 16414 raft_consensus.cc:2726] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:10:56.458113 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:56.457902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:36673)
0620 23:10:56.458042 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:56.458070 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260620 23:10:56.489599 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:56.489405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:50821)
0620 23:10:56.489521 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:56.489553 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
I20260620 23:10:56.612007 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:56.611826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:45655)
0620 23:10:56.611947 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:56.611972 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260620 23:10:57.458774 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:57.458565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:60597)
0620 23:10:57.458684 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:57.458716 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260620 23:10:57.490182 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:57.489989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:54169)
0620 23:10:57.490110 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:57.490142 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260620 23:10:57.612610 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:57.612422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:56961)
0620 23:10:57.612542 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:57.612573 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260620 23:10:58.459384 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:58.459181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:39169)
0620 23:10:58.459308 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:58.459340 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260620 23:10:58.490770 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:58.490596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:60807)
0620 23:10:58.490700 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:58.490731 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260620 23:10:58.613291 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:58.613077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:33043)
0620 23:10:58.613206 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:58.613257 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260620 23:10:59.460000 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:59.459801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:58747)
0620 23:10:59.459921 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:59.459954 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260620 23:10:59.491396 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:59.491231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:46431)
0620 23:10:59.491334 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:59.491363 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":50,"threads_started":1}
I20260620 23:10:59.613907 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:10:59.613709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:36249)
0620 23:10:59.613838 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:10:59.613868 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260620 23:11:00.417615 16427 raft_consensus.cc:482] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:11:00.417733 16427 raft_consensus.cc:504] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "820d54f23d2a4dee9b7f4b17c1808ebb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 37509 } } peers { permanent_uuid: "4fc3e3c1b42947aa919116e60bf330cc" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34437 } } peers { permanent_uuid: "287afdcbfc8e447aac28c912edef9184" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 46449 } }
I20260620 23:11:00.417945 16427 leader_election.cc:288] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509), 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437)
W20260620 23:11:00.418291 16009 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4fc3e3c1b42947aa919116e60bf330cc (127.1.32.67:34437): Not found: Tablet not found: a1f686d032854829a02af522f902da22
W20260620 23:11:00.418344 16007 leader_election.cc:341] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 820d54f23d2a4dee9b7f4b17c1808ebb (127.1.32.66:37509): Not found: Tablet not found: a1f686d032854829a02af522f902da22
I20260620 23:11:00.418370 16009 leader_election.cc:302] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 287afdcbfc8e447aac28c912edef9184; no voters: 4fc3e3c1b42947aa919116e60bf330cc, 820d54f23d2a4dee9b7f4b17c1808ebb
I20260620 23:11:00.418439 16427 raft_consensus.cc:2726] T a1f686d032854829a02af522f902da22 P 287afdcbfc8e447aac28c912edef9184 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:11:00.460588 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:00.460409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:44673)
0620 23:11:00.460519 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:00.460547 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260620 23:11:00.492026 16429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:00.491840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:52485)
0620 23:11:00.491958 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:00.491985 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260620 23:11:00.614524 16430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:00.614337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:49719)
0620 23:11:00.614455 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:00.614488 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260620 23:11:01.461194 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:01.460972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:42369)
0620 23:11:01.461115 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:01.461149 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260620 23:11:01.492609 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:01.492434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:42313)
0620 23:11:01.492548 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:01.492573 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260620 23:11:01.615214 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:01.615021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:42123)
0620 23:11:01.615138 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:01.615173 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260620 23:11:02.461972 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:02.461748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:36679)
0620 23:11:02.461879 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:02.461916 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260620 23:11:02.493288 16435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:02.493066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:50531)
0620 23:11:02.493193 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:02.493234 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260620 23:11:02.615859 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:02.615678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:39931)
0620 23:11:02.615788 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:02.615821 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260620 23:11:03.462754 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:03.462527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:54531)
0620 23:11:03.462668 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:03.462704 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260620 23:11:03.493963 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:03.493772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:58859)
0620 23:11:03.493878 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:03.493927 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260620 23:11:03.616536 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:03.616314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:60829)
0620 23:11:03.616455 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:03.616491 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260620 23:11:04.463634 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:04.463429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.65:50873)
0620 23:11:04.463550 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:04.463591 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260620 23:11:04.494799 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:04.494580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.66:41629)
0620 23:11:04.494717 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:04.494757 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260620 23:11:04.617386 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:04.617175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:45571 (local address 127.1.32.67:37035)
0620 23:11:04.617303 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:04.617346 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:45571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260620 23:11:04.620271 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 15961
I20260620 23:11:04.625144 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 16124
I20260620 23:11:04.630599 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 16257
2026-06-20T23:11:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19032 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:11:05 dist-test-slave-fzjm krb5kdc[16445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:11:05 dist-test-slave-fzjm krb5kdc[16445](info): set up 2 sockets
Jun 20 23:11:05 dist-test-slave-fzjm krb5kdc[16445](info): commencing operation
krb5kdc: starting...
W20260620 23:11:06.438762 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:11:06 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:11:06.549573 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38725
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:38725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:06.627818 16458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:06.627993 16458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:06.628013 16458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:06.629109 16458 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:11:06.629145 16458 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:11:06.629158 16458 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:11:06.629168 16458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:11:06.629179 16458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:06.629190 16458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:06.629201 16458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:11:06.629230 16458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:11:06.630596 16458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:38725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:06.630831 16458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:06.631065 16458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:06.635630 16466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:06.635654 16464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:06.635644 16463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:06.635707 16458 server_base.cc:1034] running on GCE node
I20260620 23:11:06.636130 16458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:06.636418 16458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:06.637598 16458 hybrid_clock.cc:648] HybridClock initialized: now 1781997066637585 us; error 25 us; skew 500 ppm
Jun 20 23:11:06 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:06.640018 16458 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:11:06.640465 16458 webserver.cc:460] Webserver started at http://127.1.32.126:39261/ using document root <none> and password file <none>
I20260620 23:11:06.640673 16458 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:06.640734 16458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:06.640879 16458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:11:06.641849 16458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "415ae5995ecb43b6bd3bdf556cfb8346"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "04817768fb6f2ab89559821cd490a9e4"
server_key_iv: "3ab0d6bfc11fb3e261054cc482bd9f82"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.642187 16458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "415ae5995ecb43b6bd3bdf556cfb8346"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "04817768fb6f2ab89559821cd490a9e4"
server_key_iv: "3ab0d6bfc11fb3e261054cc482bd9f82"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.643553 16458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:11:06.644251 16473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:06.644405 16458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:11:06.644503 16458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "415ae5995ecb43b6bd3bdf556cfb8346"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "04817768fb6f2ab89559821cd490a9e4"
server_key_iv: "3ab0d6bfc11fb3e261054cc482bd9f82"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.644574 16458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:06.658756 16458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:06.660588 16458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:06.660744 16458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:06.664765 16458 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:38725
I20260620 23:11:06.664794 16525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:38725 every 8 connection(s)
I20260620 23:11:06.665122 16458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:11:06.665920 16526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:11:06.668171 16526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Bootstrap starting.
I20260620 23:11:06.668715 16526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Neither blocks nor log segments found. Creating new log.
I20260620 23:11:06.668953 16526 log.cc:824] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:06.669632 16526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: No bootstrap required, opened a new log
I20260620 23:11:06.671188 16526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:06.671321 16526 raft_consensus.cc:374] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:11:06.671343 16526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415ae5995ecb43b6bd3bdf556cfb8346, State: Initialized, Role: FOLLOWER
I20260620 23:11:06.671449 16526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:06.671496 16526 raft_consensus.cc:388] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:11:06.671514 16526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:11:06.671581 16526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:11:06.672060 16526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:06.672140 16526 leader_election.cc:302] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 415ae5995ecb43b6bd3bdf556cfb8346; no voters:
I20260620 23:11:06.672369 16526 leader_election.cc:288] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:11:06.672399 16529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:11:06.672600 16529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 LEADER]: Becoming Leader. State: Replica: 415ae5995ecb43b6bd3bdf556cfb8346, State: Running, Role: LEADER
I20260620 23:11:06.672634 16526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:11:06.672830 16529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:06.673259 16530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } } }
I20260620 23:11:06.673270 16531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 415ae5995ecb43b6bd3bdf556cfb8346. Latest consensus state: current_term: 1 leader_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } } }
I20260620 23:11:06.673411 16530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: This master's current role is: LEADER
I20260620 23:11:06.673449 16531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: This master's current role is: LEADER
I20260620 23:11:06.673498 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 16458
I20260620 23:11:06.673595 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:11:06.673779 1153 external_mini_cluster.cc:1428] Setting key 2eab5d42d1450092bf73a836feba83ce
W20260620 23:11:06.674646 16544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:11:06.674871 16544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:11:06.674988 16538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:11:06.675163 16538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:11:06.678910 16538 catalog_manager.cc:1347] Generated new cluster ID: 6cba6cb26c9f4f9dacb995491d82457a
I20260620 23:11:06.678983 16538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:11:06 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:11:06.688793 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:06.674462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42956 (local address 127.1.32.126:38725)
0620 23:11:06.674969 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:06.674980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:06.675011 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:11:06.675128 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:06.675134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:06.675176 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:06.675321 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:06.676039 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:06.676984 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:06.677794 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:06.678214 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:06.678329 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:06.685131 (+ 6802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:06.685135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:06.685136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:06.685148 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:06.687144 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:06.687548 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:06.687551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:06.687551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:06.687653 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:06.687872 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:06.687873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:06.687873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:06.688153 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:06.688296 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:06.688427 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 23:11:06.688558 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":92,"threads_started":1}
I20260620 23:11:06.712519 16538 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:11:06.713064 16538 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:11:06.722401 16538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Generated new TSK 0
I20260620 23:11:06.722584 16538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:11:06.749068 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38725
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:06.827608 16552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:06.827833 16552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:06.827874 16552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:06.829027 16552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:06.829084 16552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:06.829128 16552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:11:06.830547 16552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:06.830832 16552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:06.831087 16552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:06.835628 16558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:06.835613 16557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:06.835743 16552 server_base.cc:1034] running on GCE node
W20260620 23:11:06.835628 16560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:06.836073 16552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:06.836345 16552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:06.837486 16552 hybrid_clock.cc:648] HybridClock initialized: now 1781997066837464 us; error 55 us; skew 500 ppm
Jun 20 23:11:06 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:06.839973 16552 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:11:06.840329 16552 webserver.cc:460] Webserver started at http://127.1.32.65:40449/ using document root <none> and password file <none>
I20260620 23:11:06.840509 16552 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:06.840548 16552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:06.840631 16552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:11:06.841607 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "733022fef359484abb7eefa719c683cd"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "e5c32389189276d2a8389a5e1d650ebe"
server_key_iv: "c5b5e028cac8fdaecf924c9934b6a467"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.841898 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "733022fef359484abb7eefa719c683cd"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "e5c32389189276d2a8389a5e1d650ebe"
server_key_iv: "c5b5e028cac8fdaecf924c9934b6a467"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.843191 16552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:11:06.844034 16567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:06.844316 16552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:06.844391 16552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "733022fef359484abb7eefa719c683cd"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "e5c32389189276d2a8389a5e1d650ebe"
server_key_iv: "c5b5e028cac8fdaecf924c9934b6a467"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.844444 16552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:06.857574 16552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:06.859815 16552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:06.859915 16552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:06.860076 16552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:06.860354 16552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:11:06.860386 16552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:06.860425 16552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:11:06.860450 16552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:06.865877 16552 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:36629
I20260620 23:11:06.865993 16680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:36629 every 8 connection(s)
I20260620 23:11:06.866271 16552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
Jun 20 23:11:06 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:11:06.873629 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 16552
I20260620 23:11:06.873761 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:11:06.873977 1153 external_mini_cluster.cc:1428] Setting key cfe909a332b85cf88212b074374f2494
I20260620 23:11:06.875032 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:06.866980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:53241 (local address 127.1.32.126:38725)
0620 23:11:06.867019 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:06.867021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:06.867297 (+ 276us) server_negotiation.cc:407] Connection header received
0620 23:11:06.867795 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:06.867797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:06.867812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:06.867857 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:06.868732 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:06.869221 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:06.870201 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:06.870400 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:06.870466 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:06.872350 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:06.872351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:06.872352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:06.872358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:06.873618 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:06.874096 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:06.874097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:06.874097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:06.874151 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:06.874408 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:06.874409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:06.874409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:06.874520 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:06.874584 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:06.874836 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 23:11:06.874952 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:11:06.875227 16683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:06.866999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:53241)
0620 23:11:06.867143 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:06.867208 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:06.867697 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:06.867932 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:06.867935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:06.868460 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:06.868631 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:06.868635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:06.869312 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:06.869313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:06.870108 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:06.870109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:06.870482 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:06.870483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:06.870602 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:06.871074 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:06.871081 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:06.872262 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:06.873689 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:06.873695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:06.873697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:06.874005 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:06.874213 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:06.874214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:06.874214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:06.874358 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:06.874586 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:06.874592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:06.874714 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 23:11:06.874804 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 23:11:06.874965 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260620 23:11:06.875664 16681 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38725
I20260620 23:11:06.875752 16681 heartbeater.cc:464] Registering TS with master...
I20260620 23:11:06.875917 16681 heartbeater.cc:511] Master 127.1.32.126:38725 requested a full tablet report, sending...
I20260620 23:11:06.876350 16490 ts_manager.cc:194] Registered new tserver with Master: 733022fef359484abb7eefa719c683cd (127.1.32.65:36629)
I20260620 23:11:06.876955 16490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:53241
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:11:06.895824 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38725
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:06.973444 16686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:06.973635 16686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:06.973657 16686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:06.974711 16686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:06.974751 16686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:06.974776 16686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:11:06.976089 16686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:06.976270 16686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:06.976430 16686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:06.981266 16694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:06.981351 16686 server_base.cc:1034] running on GCE node
W20260620 23:11:06.981294 16691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:06.981303 16692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:06.981637 16686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:06.981922 16686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:06.983088 16686 hybrid_clock.cc:648] HybridClock initialized: now 1781997066983075 us; error 40 us; skew 500 ppm
Jun 20 23:11:06 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:06.985471 16686 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:11:06.985890 16686 webserver.cc:460] Webserver started at http://127.1.32.66:39859/ using document root <none> and password file <none>
I20260620 23:11:06.986126 16686 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:06.986219 16686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:06.986372 16686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:11:06.987468 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "b806c891c220059c23c0616fd8031042"
server_key_iv: "07522c983af1be5aeec7008f56f79417"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.987833 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "b806c891c220059c23c0616fd8031042"
server_key_iv: "07522c983af1be5aeec7008f56f79417"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.989132 16686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:11:06.989873 16701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:06.990051 16686 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:11:06.990164 16686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "b806c891c220059c23c0616fd8031042"
server_key_iv: "07522c983af1be5aeec7008f56f79417"
server_key_version: "encryptionkey@0"
I20260620 23:11:06.990264 16686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:07.005082 16686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:07.007181 16686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:07.007349 16686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:07.007591 16686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:07.007926 16686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:11:07.007979 16686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:07.008023 16686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:11:07.008054 16686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:07.014292 16686 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:41113
I20260620 23:11:07.014343 16814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:41113 every 8 connection(s)
I20260620 23:11:07.014642 16686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:11:07.020328 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 16686
I20260620 23:11:07.020398 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
Jun 20 23:11:07 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:11:07.020612 1153 external_mini_cluster.cc:1428] Setting key 922ce2bbe80a2fb609ea4b45f2293a68
I20260620 23:11:07.024044 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.015324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:38441 (local address 127.1.32.126:38725)
0620 23:11:07.015372 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.015374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.015615 (+ 241us) server_negotiation.cc:407] Connection header received
0620 23:11:07.016143 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.016145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.016158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.016213 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:07.017112 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.017581 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.018562 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.018750 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.018832 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.020978 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:07.020979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:07.020980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:07.020986 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:07.022563 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:07.023042 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:07.023043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:07.023044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:07.023106 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:07.023382 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:07.023383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:07.023383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:07.023501 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:07.023575 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.023853 (+ 278us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.023959 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:11:07.024272 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.015308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:38441)
0620 23:11:07.015490 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.015523 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:07.016037 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:07.016288 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:07.016292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:07.016824 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:07.016998 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.017003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.017707 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.017708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.018454 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.018456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.018841 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.018842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.019016 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.019467 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:07.019475 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:07.020913 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:07.022642 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:07.022647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:07.022649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:07.022865 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:07.023197 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:07.023198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:07.023198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:07.023304 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:07.023565 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:07.023571 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:07.023707 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 23:11:07.023830 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 23:11:07.023994 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260620 23:11:07.024682 16815 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38725
I20260620 23:11:07.024792 16815 heartbeater.cc:464] Registering TS with master...
I20260620 23:11:07.024987 16815 heartbeater.cc:511] Master 127.1.32.126:38725 requested a full tablet report, sending...
I20260620 23:11:07.025377 16490 ts_manager.cc:194] Registered new tserver with Master: 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113)
I20260620 23:11:07.025755 16490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:38441
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:11:07.043779 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38725
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:07.121423 16820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:07.121599 16820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:07.121620 16820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:07.122701 16820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:07.122740 16820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:07.122766 16820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:11:07.124222 16820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:07.124457 16820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:07.124686 16820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:07.129626 16828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:07.129644 16826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:07.129621 16825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:07.129657 16820 server_base.cc:1034] running on GCE node
I20260620 23:11:07.130002 16820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:07.130208 16820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:07.131353 16820 hybrid_clock.cc:648] HybridClock initialized: now 1781997067131338 us; error 32 us; skew 500 ppm
Jun 20 23:11:07 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:07.133550 16820 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:11:07.133889 16820 webserver.cc:460] Webserver started at http://127.1.32.67:39885/ using document root <none> and password file <none>
I20260620 23:11:07.134061 16820 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:07.134097 16820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:07.134176 16820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:11:07.135205 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "8664b12b2b7b4d34b41cc4cf6088a484"
format_stamp: "Formatted at 2026-06-20 23:11:07 on dist-test-slave-fzjm"
server_key: "26de81c8a6f31af62b0fe3b6f0a8c757"
server_key_iv: "dfcd6f6057b36fa28d0ee37b9b70aad5"
server_key_version: "encryptionkey@0"
I20260620 23:11:07.135499 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "8664b12b2b7b4d34b41cc4cf6088a484"
format_stamp: "Formatted at 2026-06-20 23:11:07 on dist-test-slave-fzjm"
server_key: "26de81c8a6f31af62b0fe3b6f0a8c757"
server_key_iv: "dfcd6f6057b36fa28d0ee37b9b70aad5"
server_key_version: "encryptionkey@0"
I20260620 23:11:07.136682 16820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:11:07.137341 16835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:07.137519 16820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:11:07.137600 16820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "8664b12b2b7b4d34b41cc4cf6088a484"
format_stamp: "Formatted at 2026-06-20 23:11:07 on dist-test-slave-fzjm"
server_key: "26de81c8a6f31af62b0fe3b6f0a8c757"
server_key_iv: "dfcd6f6057b36fa28d0ee37b9b70aad5"
server_key_version: "encryptionkey@0"
I20260620 23:11:07.137663 16820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:07.147509 16820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:07.149942 16820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:07.150050 16820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:07.150230 16820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:07.150477 16820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:11:07.150504 16820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:07.150523 16820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:11:07.150534 16820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:07.156448 16820 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39197
I20260620 23:11:07.156525 16948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39197 every 8 connection(s)
I20260620 23:11:07.156857 16820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:11:07.157938 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 16820
I20260620 23:11:07.158046 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:11:07.158226 1153 external_mini_cluster.cc:1428] Setting key 0cf4abe28cd930dc0125c99cda82ed7d
Jun 20 23:11:07 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:11:07.165758 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.157550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:45047 (local address 127.1.32.126:38725)
0620 23:11:07.157630 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.157632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.157811 (+ 179us) server_negotiation.cc:407] Connection header received
0620 23:11:07.158428 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.158430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.158443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.158497 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:07.159578 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.160085 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.161016 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.161194 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.161244 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.163311 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:07.163313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:07.163313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:07.163319 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:07.164530 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:07.164952 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:07.164952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:07.164953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:07.165004 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:07.165248 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:07.165249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:07.165249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:07.165352 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:07.165406 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.165630 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.165695 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:11:07.165977 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.157518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:45047)
0620 23:11:07.157683 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.157718 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:07.158315 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:07.158566 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:07.158570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:07.159318 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:07.159502 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.159506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.160161 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.160162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.160945 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.160947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.161273 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.161274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.161368 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.161872 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:07.161879 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:07.163218 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:07.164629 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:07.164635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:07.164637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:07.164874 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:07.165088 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:07.165088 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:07.165089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:07.165190 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:07.165410 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:07.165415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:07.165524 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 23:11:07.165603 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:11:07.165719 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260620 23:11:07.166486 16949 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38725
I20260620 23:11:07.166608 16949 heartbeater.cc:464] Registering TS with master...
I20260620 23:11:07.166806 16949 heartbeater.cc:511] Master 127.1.32.126:38725 requested a full tablet report, sending...
I20260620 23:11:07.167155 16490 ts_manager.cc:194] Registered new tserver with Master: 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
I20260620 23:11:07.167533 16490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:45047
I20260620 23:11:07.169993 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:11:07.177219 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.171314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.1.32.126:38725)
0620 23:11:07.171371 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.171373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.171490 (+ 117us) server_negotiation.cc:407] Connection header received
0620 23:11:07.171539 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.171540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.171548 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.171599 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:07.172140 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.172602 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.173479 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.173727 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.173782 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.174847 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:07.174849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:07.174849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:07.174854 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:07.176287 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:07.176587 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:07.176588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:07.176588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:07.176634 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:07.176756 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:07.176757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:07.176757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:07.176871 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:07.176939 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.177096 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.177151 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 23:11:07.178759 16490 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:11:07.180269 16490 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:11:07.186430 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.182208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41113 (local address 127.0.0.1:33464)
0620 23:11:07.182355 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.182391 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:07.182451 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:07.182796 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:07.182798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:07.182800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:07.182868 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.182870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.184203 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.184204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.185241 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.185244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.186077 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.186077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.186183 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.186200 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 23:11:07.186296 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 23:11:07.186351 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":60,"thread_start_us":51,"threads_started":1}
I20260620 23:11:07.186450 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.182335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.1.32.66:41113)
0620 23:11:07.182541 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.182546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.182558 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:11:07.182602 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.182605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.182622 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.182711 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:07.182965 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.184089 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.185322 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.185993 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.186068 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.186127 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.186326 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.186372 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260620 23:11:07.186736 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.182390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39197 (local address 127.0.0.1:49664)
0620 23:11:07.182727 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.182736 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:07.182791 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:07.183075 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:07.183076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:07.183078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:07.183115 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.183117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.184296 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.184297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.185436 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.185437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.186274 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.186275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.186329 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.186332 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:11:07.186383 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 23:11:07.186414 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":4,"thread_start_us":42,"threads_started":1}
I20260620 23:11:07.187353 16749 tablet_service.cc:1467] Processing CreateTablet for tablet 944784d5021b4845a9f70c53f82558f6 (DEFAULT_TABLE table=test-table [id=1e644a3784bf48ba89aea675cffecd01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:11:07.187492 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.182215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:36629 (local address 127.0.0.1:32936)
0620 23:11:07.182580 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.182591 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:07.182645 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:07.183277 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:07.183278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:07.183279 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:07.183314 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.183315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.184157 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.184158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.185435 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.185437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.187246 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.187247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.187310 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.187314 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:11:07.187363 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:11:07.187407 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":25,"thread_start_us":59,"threads_started":1}
I20260620 23:11:07.187738 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.182423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.1.32.65:36629)
0620 23:11:07.183045 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.183050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.183059 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:11:07.183095 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.183097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.183112 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.183192 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:07.183386 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.184056 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.186366 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.187175 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.187245 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.187297 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.187613 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.187659 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":70,"threads_started":1}
I20260620 23:11:07.187755 16749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944784d5021b4845a9f70c53f82558f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:11:07.190138 16963 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Bootstrap starting.
I20260620 23:11:07.190537 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.182548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.1.32.67:39197)
0620 23:11:07.182724 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.182729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.182779 (+ 50us) server_negotiation.cc:407] Connection header received
0620 23:11:07.182851 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.182853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.182866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.182996 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:07.183199 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.184202 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.185518 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.186206 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.186258 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.186325 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.186486 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.190420 (+ 3934us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260620 23:11:07.190815 16963 tablet_bootstrap.cc:654] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Neither blocks nor log segments found. Creating new log.
I20260620 23:11:07.190797 16615 tablet_service.cc:1467] Processing CreateTablet for tablet 944784d5021b4845a9f70c53f82558f6 (DEFAULT_TABLE table=test-table [id=1e644a3784bf48ba89aea675cffecd01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:11:07.191197 16615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944784d5021b4845a9f70c53f82558f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:11:07.191200 16883 tablet_service.cc:1467] Processing CreateTablet for tablet 944784d5021b4845a9f70c53f82558f6 (DEFAULT_TABLE table=test-table [id=1e644a3784bf48ba89aea675cffecd01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:11:07.191428 16963 log.cc:824] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:07.191529 16883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944784d5021b4845a9f70c53f82558f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:11:07.192185 16963 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: No bootstrap required, opened a new log
I20260620 23:11:07.192301 16963 ts_tablet_manager.cc:1397] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 23:11:07.194451 16965 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Bootstrap starting.
I20260620 23:11:07.194440 16966 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Bootstrap starting.
I20260620 23:11:07.194430 16963 raft_consensus.cc:348] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.194630 16963 raft_consensus.cc:374] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:11:07.194684 16963 raft_consensus.cc:729] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487f0c4fd6b24fbc8cc61cf2e00ddd59, State: Initialized, Role: FOLLOWER
I20260620 23:11:07.194832 16963 consensus_queue.cc:260] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.195163 16966 tablet_bootstrap.cc:654] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Neither blocks nor log segments found. Creating new log.
I20260620 23:11:07.195176 16815 heartbeater.cc:503] Master 127.1.32.126:38725 was elected leader, sending a full tablet report...
I20260620 23:11:07.195171 16963 ts_tablet_manager.cc:1428] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:11:07.195605 16965 tablet_bootstrap.cc:654] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Neither blocks nor log segments found. Creating new log.
I20260620 23:11:07.195762 16966 log.cc:824] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:07.196225 16965 log.cc:824] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:07.196651 16966 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: No bootstrap required, opened a new log
I20260620 23:11:07.196735 16966 ts_tablet_manager.cc:1397] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:11:07.196938 16965 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: No bootstrap required, opened a new log
I20260620 23:11:07.197006 16965 ts_tablet_manager.cc:1397] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 23:11:07.198678 16966 raft_consensus.cc:348] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.198943 16966 raft_consensus.cc:374] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:11:07.198993 16966 raft_consensus.cc:729] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733022fef359484abb7eefa719c683cd, State: Initialized, Role: FOLLOWER
I20260620 23:11:07.198896 16965 raft_consensus.cc:348] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.199097 16965 raft_consensus.cc:374] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:11:07.199141 16965 raft_consensus.cc:729] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8664b12b2b7b4d34b41cc4cf6088a484, State: Initialized, Role: FOLLOWER
I20260620 23:11:07.199098 16966 consensus_queue.cc:260] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.199259 16965 consensus_queue.cc:260] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.199361 16966 ts_tablet_manager.cc:1428] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260620 23:11:07.199491 16681 heartbeater.cc:503] Master 127.1.32.126:38725 was elected leader, sending a full tablet report...
I20260620 23:11:07.199514 16965 ts_tablet_manager.cc:1428] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260620 23:11:07.199559 16949 heartbeater.cc:503] Master 127.1.32.126:38725 was elected leader, sending a full tablet report...
W20260620 23:11:07.265218 16816 tablet.cc:2369] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:11:07.367063 16682 tablet.cc:2369] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:11:07.380872 16970 raft_consensus.cc:482] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:11:07.380978 16970 raft_consensus.cc:504] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.381345 16970 leader_election.cc:288] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
I20260620 23:11:07.385392 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.381614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:43053 (local address 127.1.32.66:41113)
0620 23:11:07.381667 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.381669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.381864 (+ 195us) server_negotiation.cc:407] Connection header received
0620 23:11:07.381903 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.381904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.381916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.381985 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:07.382213 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.383003 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.384465 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.385029 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.385111 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.385121 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.385277 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.385316 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 23:11:07.385392 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.381661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:57459 (local address 127.1.32.67:39197)
0620 23:11:07.381720 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.381725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.381875 (+ 150us) server_negotiation.cc:407] Connection header received
0620 23:11:07.381904 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.381905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.381918 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.381985 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:07.382210 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.383003 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.384465 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.385040 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.385111 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.385121 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.385289 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.385327 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:11:07.385421 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.381543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41113 (local address 127.1.32.65:43053)
0620 23:11:07.381721 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.381733 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:07.381832 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:07.382001 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:07.382002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:07.382004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:07.382114 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.382116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.383086 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.383086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.384369 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.384372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.385118 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.385118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.385197 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.385204 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 23:11:07.385258 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:11:07.385307 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":45,"thread_start_us":103,"threads_started":1}
I20260620 23:11:07.385421 16973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.381543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39197 (local address 127.1.32.65:57459)
0620 23:11:07.381722 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.381733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:07.381832 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:07.382001 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:07.382002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:07.382006 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:07.382111 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.382113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.383088 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.383088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.384369 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:07.384372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.385116 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.385117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:07.385197 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.385202 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:11:07.385271 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:11:07.385307 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":156,"thread_start_us":52,"threads_started":1}
I20260620 23:11:07.385982 16769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944784d5021b4845a9f70c53f82558f6" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" is_pre_election: true
I20260620 23:11:07.386008 16903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944784d5021b4845a9f70c53f82558f6" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" is_pre_election: true
I20260620 23:11:07.386144 16903 raft_consensus.cc:2454] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 0.
I20260620 23:11:07.386148 16769 raft_consensus.cc:2454] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 0.
I20260620 23:11:07.386406 16571 leader_election.cc:302] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 733022fef359484abb7eefa719c683cd, 8664b12b2b7b4d34b41cc4cf6088a484; no voters:
I20260620 23:11:07.386555 16970 raft_consensus.cc:2781] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:11:07.386608 16970 raft_consensus.cc:482] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:11:07.386637 16970 raft_consensus.cc:3037] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:11:07.387394 16970 raft_consensus.cc:504] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.387539 16970 leader_election.cc:288] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 election: Requested vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
I20260620 23:11:07.387758 16769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944784d5021b4845a9f70c53f82558f6" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59"
I20260620 23:11:07.387794 16903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944784d5021b4845a9f70c53f82558f6" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8664b12b2b7b4d34b41cc4cf6088a484"
I20260620 23:11:07.387857 16769 raft_consensus.cc:3037] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:11:07.387853 16903 raft_consensus.cc:3037] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:11:07.388525 16769 raft_consensus.cc:2454] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 1.
I20260620 23:11:07.388576 16903 raft_consensus.cc:2454] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 1.
I20260620 23:11:07.388762 16568 leader_election.cc:302] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 733022fef359484abb7eefa719c683cd; no voters:
I20260620 23:11:07.388882 16970 raft_consensus.cc:2781] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:11:07.389081 16970 raft_consensus.cc:686] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 1 LEADER]: Becoming Leader. State: Replica: 733022fef359484abb7eefa719c683cd, State: Running, Role: LEADER
I20260620 23:11:07.389180 16970 consensus_queue.cc:237] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:07.389993 16489 catalog_manager.cc:5495] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 733022fef359484abb7eefa719c683cd (127.1.32.65). New cstate: current_term: 1 leader_uuid: "733022fef359484abb7eefa719c683cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } health_report { overall_health: UNKNOWN } } }
W20260620 23:11:07.407374 16950 tablet.cc:2369] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:11:07.412175 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.410006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.1.32.65:36629)
0620 23:11:07.410065 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:07.410067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:07.410077 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:11:07.410119 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:07.410120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:07.410138 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:07.410216 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:11:07.410348 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.410808 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.411494 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:07.411672 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:07.411725 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:07.411881 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:11:07.412006 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:11:07.412068 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:07.412099 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 23:11:07.412118 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:11:07.413897 16769 raft_consensus.cc:1261] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Refusing update from remote peer 733022fef359484abb7eefa719c683cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:11:07.414268 16903 raft_consensus.cc:1261] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Refusing update from remote peer 733022fef359484abb7eefa719c683cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:11:07.414316 16974 consensus_queue.cc:1035] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:11:07.414564 16974 consensus_queue.cc:1035] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:11:07.417456 16978 mvcc.cc:205] Tried to move back new op lower bound from 7299059988124745728 to 7299059988026671104. Current Snapshot: MvccSnapshot[applied={T|T < 7299059988124745728}]
I20260620 23:11:07.417780 16979 mvcc.cc:205] Tried to move back new op lower bound from 7299059988124745728 to 7299059988026671104. Current Snapshot: MvccSnapshot[applied={T|T < 7299059988124745728}]
I20260620 23:11:07.418692 16490 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:11:07.418790 16490 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:11:07.419513 16490 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Sending DeleteTablet for 3 replicas of tablet 944784d5021b4845a9f70c53f82558f6
I20260620 23:11:07.419852 16749 tablet_service.cc:1514] Processing DeleteTablet for tablet 944784d5021b4845a9f70c53f82558f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:11:07 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:33464
I20260620 23:11:07.419857 16615 tablet_service.cc:1514] Processing DeleteTablet for tablet 944784d5021b4845a9f70c53f82558f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:11:07 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:32936
I20260620 23:11:07.419893 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 16552
I20260620 23:11:07.420058 16883 tablet_service.cc:1514] Processing DeleteTablet for tablet 944784d5021b4845a9f70c53f82558f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:11:07 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:49664
I20260620 23:11:07.420699 16987 tablet_replica.cc:330] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: stopping tablet replica
I20260620 23:11:07.420809 16987 raft_consensus.cc:2229] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:11:07.420912 16987 raft_consensus.cc:2258] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:11:07.421283 16987 ts_tablet_manager.cc:1905] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:11:07.423316 16987 ts_tablet_manager.cc:1918] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:11:07.423377 16987 log.cc:1196] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/944784d5021b4845a9f70c53f82558f6
I20260620 23:11:07.423645 16987 ts_tablet_manager.cc:1939] T 944784d5021b4845a9f70c53f82558f6 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Deleting consensus metadata
I20260620 23:11:07.424045 16474 catalog_manager.cc:4841] TS 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): tablet 944784d5021b4845a9f70c53f82558f6 (table test-table [id=1e644a3784bf48ba89aea675cffecd01]) successfully deleted
I20260620 23:11:07.426385 16988 tablet_replica.cc:330] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: stopping tablet replica
W20260620 23:11:07.426442 16477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:36629 (error 108)
I20260620 23:11:07.426544 16988 raft_consensus.cc:2229] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:11:07.426577 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 16686
I20260620 23:11:07.426668 16988 raft_consensus.cc:2258] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:11:07.427120 16988 ts_tablet_manager.cc:1905] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:11:07.427665 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:07.426805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:36629 (local address 127.0.0.1:32956)
0620 23:11:07.427600 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:07.427638 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":775}
W20260620 23:11:07.427805 16477 catalog_manager.cc:4584] TS 733022fef359484abb7eefa719c683cd (127.1.32.65:36629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 944784d5021b4845a9f70c53f82558f6: Network error: Client connection negotiation failed: client connection to 127.1.32.65:36629: connect: Connection refused (error 111)
I20260620 23:11:07.428412 16988 ts_tablet_manager.cc:1918] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:11:07.428498 16988 log.cc:1196] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/944784d5021b4845a9f70c53f82558f6
I20260620 23:11:07.428771 16988 ts_tablet_manager.cc:1939] T 944784d5021b4845a9f70c53f82558f6 P 8664b12b2b7b4d34b41cc4cf6088a484: Deleting consensus metadata
I20260620 23:11:07.429170 16477 catalog_manager.cc:4841] TS 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): tablet 944784d5021b4845a9f70c53f82558f6 (table test-table [id=1e644a3784bf48ba89aea675cffecd01]) successfully deleted
I20260620 23:11:07.432332 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 16820
I20260620 23:11:07.437000 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 16458
I20260620 23:11:27.442343 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38725
--webserver_interface=127.1.32.126
--webserver_port=39261
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:38725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:27.518317 16990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:27.518509 16990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:27.518532 16990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:27.519687 16990 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:11:27.519724 16990 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:11:27.519737 16990 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:11:27.519749 16990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:11:27.519763 16990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:27.519775 16990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:27.519827 16990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:11:27.519855 16990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:11:27.521183 16990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:38725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:27.521405 16990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:27.521665 16990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:27.531016 16996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.531111 16997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.531153 16999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:27.531297 16990 server_base.cc:1034] running on GCE node
I20260620 23:11:27.531574 16990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:27.531903 16990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:27.533097 16990 hybrid_clock.cc:648] HybridClock initialized: now 1781997087533093 us; error 41 us; skew 500 ppm
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:27.535723 16990 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:11:27.536207 16990 webserver.cc:460] Webserver started at http://127.1.32.126:39261/ using document root <none> and password file <none>
I20260620 23:11:27.536459 16990 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:27.536556 16990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:27.537981 16990 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:27.538702 17006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.538882 16990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:11:27.539037 16990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "415ae5995ecb43b6bd3bdf556cfb8346"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "04817768fb6f2ab89559821cd490a9e4"
server_key_iv: "3ab0d6bfc11fb3e261054cc482bd9f82"
server_key_version: "encryptionkey@0"
I20260620 23:11:27.539479 16990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:27.546552 16990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:27.548341 16990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:27.548503 16990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:27.552325 16990 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:38725
I20260620 23:11:27.552350 17058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:38725 every 8 connection(s)
I20260620 23:11:27.552731 16990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:11:27.554171 17059 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:11:27.554877 17059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Bootstrap starting.
I20260620 23:11:27.556974 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 16990
I20260620 23:11:27.557267 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:36629
--local_ip_for_outbound_sockets=127.1.32.65
--tserver_master_addrs=127.1.32.126:38725
--webserver_port=40449
--webserver_interface=127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:11:27.563441 17059 log.cc:824] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:27.565172 17059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:11:27.565439 17059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Bootstrap complete.
I20260620 23:11:27.567700 17059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:27.567896 17059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415ae5995ecb43b6bd3bdf556cfb8346, State: Initialized, Role: FOLLOWER
I20260620 23:11:27.568074 17059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:27.568135 17059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:11:27.568223 17059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:11:27.568274 17059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:11:27.569260 17059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:27.569355 17059 leader_election.cc:302] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 415ae5995ecb43b6bd3bdf556cfb8346; no voters:
I20260620 23:11:27.569604 17059 leader_election.cc:288] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:11:27.569631 17063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:11:27.569744 17063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 LEADER]: Becoming Leader. State: Replica: 415ae5995ecb43b6bd3bdf556cfb8346, State: Running, Role: LEADER
I20260620 23:11:27.569828 17063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:27.569864 17059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:11:27.570072 17063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 415ae5995ecb43b6bd3bdf556cfb8346. Latest consensus state: current_term: 2 leader_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } } }
I20260620 23:11:27.570134 17063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: This master's current role is: LEADER
I20260620 23:11:27.570330 17064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } } }
I20260620 23:11:27.570392 17067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:11:27.571282 17067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:11:27.571378 17067 catalog_manager.cc:1259] Loaded cluster ID: 6cba6cb26c9f4f9dacb995491d82457a
I20260620 23:11:27.571419 17067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:11:27.572623 17067 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:11:27.572799 17067 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Loaded TSK: 0
I20260620 23:11:27.574981 17064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: This master's current role is: LEADER
I20260620 23:11:27.588034 17067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Generated new TSK 1
I20260620 23:11:27.588135 17067 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:11:27.645015 17061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:27.645277 17061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:27.645316 17061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:27.646509 17061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:27.646588 17061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:27.646636 17061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:11:27.648159 17061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:36629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=40449
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:27.648433 17061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:27.648681 17061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:27.653862 17087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.653872 17089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.653877 17086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:27.654146 17061 server_base.cc:1034] running on GCE node
I20260620 23:11:27.654500 17061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:27.654779 17061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:27.655974 17061 hybrid_clock.cc:648] HybridClock initialized: now 1781997087655961 us; error 28 us; skew 500 ppm
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:27.659046 17061 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:11:27.659493 17061 webserver.cc:460] Webserver started at http://127.1.32.65:40449/ using document root <none> and password file <none>
I20260620 23:11:27.659756 17061 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:27.659844 17061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:27.661271 17061 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:27.662034 17096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.662261 17061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:27.662361 17061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "733022fef359484abb7eefa719c683cd"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "e5c32389189276d2a8389a5e1d650ebe"
server_key_iv: "c5b5e028cac8fdaecf924c9934b6a467"
server_key_version: "encryptionkey@0"
I20260620 23:11:27.662722 17061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:27.683169 17061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:27.685566 17061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:27.685731 17061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:27.685966 17061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:27.686477 17103 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:11:27.688016 17061 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:11:27.688064 17061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 23:11:27.688102 17061 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:11:27.688769 17061 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:11:27.688813 17061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:11:27.688858 17103 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Bootstrap starting.
I20260620 23:11:27.696291 17061 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:36629
I20260620 23:11:27.696508 17210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:36629 every 8 connection(s)
I20260620 23:11:27.696805 17061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:11:27.699959 17103 log.cc:824] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:27.702507 17103 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:11:27.702643 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 17061
I20260620 23:11:27.702858 17103 tablet_bootstrap.cc:492] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Bootstrap complete.
I20260620 23:11:27.702992 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:41113
--local_ip_for_outbound_sockets=127.1.32.66
--tserver_master_addrs=127.1.32.126:38725
--webserver_port=39859
--webserver_interface=127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:11:27.704115 17103 ts_tablet_manager.cc:1397] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:11:27.705258 17103 raft_consensus.cc:348] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:27.705520 17103 raft_consensus.cc:729] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733022fef359484abb7eefa719c683cd, State: Initialized, Role: FOLLOWER
I20260620 23:11:27.705631 17103 consensus_queue.cc:260] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } } peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } }
I20260620 23:11:27.705864 17103 ts_tablet_manager.cc:1428] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260620 23:11:27.710130 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.697649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:39895 (local address 127.1.32.126:38725)
0620 23:11:27.699046 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.699056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.699105 (+ 49us) server_negotiation.cc:407] Connection header received
0620 23:11:27.699269 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.699279 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.699351 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.699516 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:27.700393 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.701308 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.702398 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.703151 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.703247 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.706028 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:27.706032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:27.706033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:27.706044 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:27.708155 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.708708 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.708711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.708712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.708825 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.709119 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.709120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.709121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.709443 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:27.709509 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.709901 (+ 392us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.710000 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1334,"thread_start_us":82,"threads_started":1}
I20260620 23:11:27.710207 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.697485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:39895)
0620 23:11:27.697771 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:27.697811 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:27.698493 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:27.699594 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:27.699599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:27.699976 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:27.700254 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.700260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.701462 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.701463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.702282 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.702284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.703355 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.703357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.703529 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.704185 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:27.704196 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:27.705941 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:27.708276 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.708282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.708284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.708613 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.708896 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.708897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.708897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.709023 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.709559 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:27.709566 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:27.709715 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 23:11:27.709827 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 23:11:27.709993 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260620 23:11:27.711129 17211 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38725
I20260620 23:11:27.711210 17211 heartbeater.cc:464] Registering TS with master...
I20260620 23:11:27.711381 17211 heartbeater.cc:511] Master 127.1.32.126:38725 requested a full tablet report, sending...
I20260620 23:11:27.712150 17023 ts_manager.cc:194] Registered new tserver with Master: 733022fef359484abb7eefa719c683cd (127.1.32.65:36629)
I20260620 23:11:27.713358 17023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:39895
I20260620 23:11:27.714367 17211 heartbeater.cc:503] Master 127.1.32.126:38725 was elected leader, sending a full tablet report...
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.65@KRBTEST.COM
I20260620 23:11:27.720539 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.712783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:36629 (local address 127.0.0.1:36152)
0620 23:11:27.713024 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:27.713046 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:27.713164 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:27.713371 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:27.713376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:27.713697 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:27.713750 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.713757 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.714623 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.714624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.715344 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.715346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.715818 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.715819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.715895 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.716363 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:27.716367 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:27.717697 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:27.719425 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.719432 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.719433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.719628 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.719870 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.719871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.719871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.719927 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.720199 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:27.720205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:27.720251 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 23:11:27.720348 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 23:11:27.720454 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
I20260620 23:11:27.720538 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.712849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36152 (local address 127.1.32.65:36629)
0620 23:11:27.712979 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.712982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.713151 (+ 169us) server_negotiation.cc:407] Connection header received
0620 23:11:27.713266 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.713269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.713288 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.713368 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:27.713837 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.714520 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.715441 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.715721 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.715816 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.717803 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:27.717810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:27.717810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:27.717816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:27.719317 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.719729 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.719732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.719732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.719800 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.719996 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.719997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.719998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.720125 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:27.720215 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.720381 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.720460 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260620 23:11:27.720927 17145 tablet_service.cc:1514] Processing DeleteTablet for tablet 944784d5021b4845a9f70c53f82558f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:11:07 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:36152
I20260620 23:11:27.721427 17220 tablet_replica.cc:330] stopping tablet replica
I20260620 23:11:27.721557 17220 raft_consensus.cc:2229] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:11:27.721647 17220 raft_consensus.cc:2258] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:11:27.721894 17220 ts_tablet_manager.cc:1905] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:11:27.723863 17220 ts_tablet_manager.cc:1918] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:11:27.723961 17220 log.cc:1196] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal/wals/944784d5021b4845a9f70c53f82558f6
I20260620 23:11:27.724157 17220 ts_tablet_manager.cc:1939] T 944784d5021b4845a9f70c53f82558f6 P 733022fef359484abb7eefa719c683cd: Deleting consensus metadata
I20260620 23:11:27.724637 17010 catalog_manager.cc:4841] TS 733022fef359484abb7eefa719c683cd (127.1.32.65:36629): tablet 944784d5021b4845a9f70c53f82558f6 (table test-table [id=1e644a3784bf48ba89aea675cffecd01]) successfully deleted
W20260620 23:11:27.787384 17216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:27.787560 17216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:27.787582 17216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:27.788671 17216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:27.788709 17216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:27.788733 17216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:11:27.790083 17216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:41113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=39859
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:27.790287 17216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:27.790511 17216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:11:27.795441 17216 server_base.cc:1034] running on GCE node
W20260620 23:11:27.795403 17226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.795413 17228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.795413 17225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:27.795758 17216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:27.796048 17216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:27.797240 17216 hybrid_clock.cc:648] HybridClock initialized: now 1781997087797223 us; error 27 us; skew 500 ppm
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:27.799616 17216 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:11:27.800081 17216 webserver.cc:460] Webserver started at http://127.1.32.66:39859/ using document root <none> and password file <none>
I20260620 23:11:27.800314 17216 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:27.800405 17216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:27.801818 17216 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:11:27.802455 17235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.802630 17216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:11:27.802735 17216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "b806c891c220059c23c0616fd8031042"
server_key_iv: "07522c983af1be5aeec7008f56f79417"
server_key_version: "encryptionkey@0"
I20260620 23:11:27.803203 17216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:27.811784 17216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:27.813577 17216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:27.813759 17216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:27.813975 17216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:27.814271 17216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:11:27.814332 17216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.814373 17216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:11:27.814402 17216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.820499 17216 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:41113
I20260620 23:11:27.820528 17348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:41113 every 8 connection(s)
I20260620 23:11:27.820858 17216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:11:27.829524 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.821585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:42169 (local address 127.1.32.126:38725)
0620 23:11:27.821673 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.821675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.821870 (+ 195us) server_negotiation.cc:407] Connection header received
0620 23:11:27.822317 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.822318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.822333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.822391 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:27.823360 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.823858 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.824767 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.824947 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.825006 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.827132 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:27.827134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:27.827134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:27.827153 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:27.828334 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.828681 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.828681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.828682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.828728 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.828927 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.828928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.828928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.829017 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:27.829084 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.829342 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.829418 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 23:11:27.829593 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.821553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:42169)
0620 23:11:27.821740 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:27.821771 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:27.822228 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:27.822438 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:27.822441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:27.823096 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:27.823272 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.823277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.823958 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.823958 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.824684 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.824685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.825023 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.825024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.825109 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.825590 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:27.825598 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:27.826988 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:27.828407 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.828413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.828415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.828617 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.828786 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.828787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.828787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.828875 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.829109 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:27.829113 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:27.829224 (+ 111us) client_negotiation.cc:769] Sending connection context
0620 23:11:27.829297 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:11:27.829430 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260620 23:11:27.829690 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 17216
I20260620 23:11:27.829912 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:39197
--local_ip_for_outbound_sockets=127.1.32.67
--tserver_master_addrs=127.1.32.126:38725
--webserver_port=39885
--webserver_interface=127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:11:27.830390 17349 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38725
I20260620 23:11:27.830476 17349 heartbeater.cc:464] Registering TS with master...
I20260620 23:11:27.830654 17349 heartbeater.cc:511] Master 127.1.32.126:38725 requested a full tablet report, sending...
I20260620 23:11:27.831058 17023 ts_manager.cc:194] Registered new tserver with Master: 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113)
I20260620 23:11:27.831413 17023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:42169
W20260620 23:11:27.909389 17352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:27.909618 17352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:27.909657 17352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:27.910733 17352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:27.910799 17352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:27.910841 17352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:11:27.912182 17352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:39197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=39885
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:27.912423 17352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:27.912640 17352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:11:27.917490 17352 server_base.cc:1034] running on GCE node
W20260620 23:11:27.917459 17357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.917388 17358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:27.917387 17360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:27.917855 17352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:27.918093 17352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:27.919279 17352 hybrid_clock.cc:648] HybridClock initialized: now 1781997087919274 us; error 39 us; skew 500 ppm
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:27.921741 17352 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:11:27.922147 17352 webserver.cc:460] Webserver started at http://127.1.32.67:39885/ using document root <none> and password file <none>
I20260620 23:11:27.922364 17352 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:27.922463 17352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:27.923887 17352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:27.924587 17367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.924820 17352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:27.924957 17352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "8664b12b2b7b4d34b41cc4cf6088a484"
format_stamp: "Formatted at 2026-06-20 23:11:07 on dist-test-slave-fzjm"
server_key: "26de81c8a6f31af62b0fe3b6f0a8c757"
server_key_iv: "dfcd6f6057b36fa28d0ee37b9b70aad5"
server_key_version: "encryptionkey@0"
I20260620 23:11:27.925316 17352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:27.942309 17352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:27.944834 17352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:27.945014 17352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:27.945248 17352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:27.945554 17352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:11:27.945616 17352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.945660 17352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:11:27.945691 17352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:27.952400 17352 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39197
I20260620 23:11:27.952490 17480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39197 every 8 connection(s)
I20260620 23:11:27.952804 17352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:11:27.954907 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 17352
Jun 20 23:11:27 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:11:27.964476 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.953596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:50641 (local address 127.1.32.126:38725)
0620 23:11:27.953653 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.953655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.953874 (+ 219us) server_negotiation.cc:407] Connection header received
0620 23:11:27.954393 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.954394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.954408 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.954472 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:27.955820 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.956682 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.957746 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.958113 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.958193 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.960638 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:27.960639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:27.960640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:27.960647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:27.962504 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.963314 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.963315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.963315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.963378 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.963713 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.963714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.963715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.963853 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:27.963929 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.964299 (+ 370us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.964377 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:11:27.964671 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.953581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:50641)
0620 23:11:27.953750 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:27.953784 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:27.954280 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:27.954582 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:27.954585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:27.955445 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:27.955719 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.955726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.956783 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.956784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.957655 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.957658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.958219 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.958220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.958369 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.958968 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:27.958976 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:27.960498 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:27.962849 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.962856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.962859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.963226 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.963472 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:27.963473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:27.963473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:27.963615 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:27.963935 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:27.963941 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:27.964131 (+ 190us) client_negotiation.cc:769] Sending connection context
0620 23:11:27.964241 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 23:11:27.964412 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260620 23:11:27.965174 17481 heartbeater.cc:347] Connected to a master server at 127.1.32.126:38725
I20260620 23:11:27.965283 17481 heartbeater.cc:464] Registering TS with master...
I20260620 23:11:27.965411 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.955340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.1.32.126:38725)
0620 23:11:27.955461 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.955463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.955647 (+ 184us) server_negotiation.cc:407] Connection header received
0620 23:11:27.955703 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.955704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.955718 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.955797 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:27.957666 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.958247 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.959502 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.959765 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.959845 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.961318 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:27.961319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:27.961319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:27.961326 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:27.963436 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.963851 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.963852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.963852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.963907 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:27.964131 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:27.964132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:27.964133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:27.964279 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:27.964444 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.965242 (+ 798us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.965321 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260620 23:11:27.965463 17481 heartbeater.cc:511] Master 127.1.32.126:38725 requested a full tablet report, sending...
I20260620 23:11:27.965865 17022 ts_manager.cc:194] Registered new tserver with Master: 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
I20260620 23:11:27.966501 17022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:50641
I20260620 23:11:27.966518 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:11:27.970192 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.966863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.1.32.126:38725)
0620 23:11:27.966950 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.966951 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.967103 (+ 152us) server_negotiation.cc:407] Connection header received
0620 23:11:27.967148 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.967149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.967185 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.967264 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:11:27.967403 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.967862 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.969092 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.969329 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.969453 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.969696 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:11:27.969836 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:11:27.969967 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.970064 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.970088 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:11:27.970505 17022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:11:27.972411 17022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:11:27.975090 17145 tablet_service.cc:1467] Processing CreateTablet for tablet d5830027b4c84dc690defd1b3a2ec518 (DEFAULT_TABLE table=test-table [id=4221ffa49eaf43ecb0ca9c0c7f9f6bd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:11:27.975291 17145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5830027b4c84dc690defd1b3a2ec518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:11:27.977232 17103 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Bootstrap starting.
I20260620 23:11:27.977560 17103 tablet_bootstrap.cc:654] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Neither blocks nor log segments found. Creating new log.
I20260620 23:11:27.978533 17103 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: No bootstrap required, opened a new log
I20260620 23:11:27.978614 17103 ts_tablet_manager.cc:1397] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 23:11:27.978786 17103 raft_consensus.cc:348] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:27.978878 17103 raft_consensus.cc:374] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:11:27.978947 17103 raft_consensus.cc:729] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733022fef359484abb7eefa719c683cd, State: Initialized, Role: FOLLOWER
I20260620 23:11:27.979029 17103 consensus_queue.cc:260] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:27.979126 17103 ts_tablet_manager.cc:1428] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 23:11:27.981865 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.974459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.1.32.67:39197)
0620 23:11:27.974637 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.974641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.974859 (+ 218us) server_negotiation.cc:407] Connection header received
0620 23:11:27.977209 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.977212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.977230 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.977319 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:27.977559 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.978854 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.980515 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.981392 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.981443 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.981503 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.981756 (+ 253us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.981792 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260620 23:11:27.981902 17489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.974357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39197 (local address 127.0.0.1:50986)
0620 23:11:27.974768 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:27.974776 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:27.974862 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:27.977427 (+ 2565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:27.977428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:27.977430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:27.977477 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.977479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.978992 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.978993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.980404 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.980407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.981570 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.981571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.981648 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.981671 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 23:11:27.981730 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 23:11:27.981779 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":283,"thread_start_us":77,"threads_started":1}
I20260620 23:11:27.982851 17415 tablet_service.cc:1467] Processing CreateTablet for tablet d5830027b4c84dc690defd1b3a2ec518 (DEFAULT_TABLE table=test-table [id=4221ffa49eaf43ecb0ca9c0c7f9f6bd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:11:27.983274 17415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5830027b4c84dc690defd1b3a2ec518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:11:27.985954 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.974345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41113 (local address 127.0.0.1:58452)
0620 23:11:27.974381 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:27.974391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:27.974431 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:27.983337 (+ 8906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:27.983338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:27.983339 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:27.983381 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.983382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.984348 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.984349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.985003 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:27.985005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.985668 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.985669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:27.985738 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.985742 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:11:27.985844 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 23:11:27.985890 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 23:11:27.985983 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:27.974501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.1.32.66:41113)
0620 23:11:27.983069 (+ 8568us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:27.983096 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:27.983109 (+ 13us) server_negotiation.cc:407] Connection header received
0620 23:11:27.983150 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:27.983157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:27.983173 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:27.983502 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:27.983597 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.984234 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.985104 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:27.985563 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:27.985664 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:27.985722 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:27.985862 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 23:11:27.985903 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8510,"thread_start_us":86,"threads_started":1}
I20260620 23:11:27.986562 17283 tablet_service.cc:1467] Processing CreateTablet for tablet d5830027b4c84dc690defd1b3a2ec518 (DEFAULT_TABLE table=test-table [id=4221ffa49eaf43ecb0ca9c0c7f9f6bd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:11:27.987021 17283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5830027b4c84dc690defd1b3a2ec518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:11:27.987569 17491 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Bootstrap starting.
I20260620 23:11:27.988334 17491 tablet_bootstrap.cc:654] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Neither blocks nor log segments found. Creating new log.
I20260620 23:11:27.988956 17491 log.cc:824] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:27.989780 17491 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: No bootstrap required, opened a new log
I20260620 23:11:27.989873 17491 ts_tablet_manager.cc:1397] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:11:27.990859 17493 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Bootstrap starting.
I20260620 23:11:27.991627 17493 tablet_bootstrap.cc:654] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Neither blocks nor log segments found. Creating new log.
I20260620 23:11:27.992261 17493 log.cc:824] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:27.992259 17491 raft_consensus.cc:348] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:27.992446 17491 raft_consensus.cc:374] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:11:27.992493 17491 raft_consensus.cc:729] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8664b12b2b7b4d34b41cc4cf6088a484, State: Initialized, Role: FOLLOWER
I20260620 23:11:27.992610 17491 consensus_queue.cc:260] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:27.992889 17491 ts_tablet_manager.cc:1428] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:11:27.992930 17481 heartbeater.cc:503] Master 127.1.32.126:38725 was elected leader, sending a full tablet report...
I20260620 23:11:27.993570 17493 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: No bootstrap required, opened a new log
I20260620 23:11:27.993644 17493 ts_tablet_manager.cc:1397] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 23:11:27.995214 17493 raft_consensus.cc:348] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:27.995368 17493 raft_consensus.cc:374] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:11:27.995414 17493 raft_consensus.cc:729] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487f0c4fd6b24fbc8cc61cf2e00ddd59, State: Initialized, Role: FOLLOWER
I20260620 23:11:27.995558 17493 consensus_queue.cc:260] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:27.995831 17493 ts_tablet_manager.cc:1428] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:11:27.995893 17349 heartbeater.cc:503] Master 127.1.32.126:38725 was elected leader, sending a full tablet report...
I20260620 23:11:28.038056 17217 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:11:28.038224 17217 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:28.038638 17217 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
I20260620 23:11:28.041795 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.039080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:45143 (local address 127.1.32.66:41113)
0620 23:11:28.039138 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:28.039140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:28.039150 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:11:28.039180 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:28.039181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:28.039194 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:28.039253 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:28.039486 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.039996 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.041023 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.041415 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.041467 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:28.041475 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:28.041668 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 23:11:28.041712 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:11:28.041802 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.038793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41113 (local address 127.1.32.65:45143)
0620 23:11:28.038871 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:28.038887 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:28.038963 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:28.039308 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:28.039310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:28.039313 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:28.039399 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:28.039401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.040084 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.040085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:28.040915 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:28.040917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.041490 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.041491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:28.041570 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:28.041593 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 23:11:28.041644 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 23:11:28.041683 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":28}
I20260620 23:11:28.041890 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.039001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:37007 (local address 127.1.32.67:39197)
0620 23:11:28.039063 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:28.039065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:28.039282 (+ 217us) server_negotiation.cc:407] Connection header received
0620 23:11:28.039314 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:28.039314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:28.039328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:28.039400 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:11:28.039601 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.040067 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.041017 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.041497 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.041568 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:28.041577 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:28.041782 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 23:11:28.041822 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:11:28.042093 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.038935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39197 (local address 127.1.32.65:37007)
0620 23:11:28.039154 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:28.039164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:28.039222 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:28.039410 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:28.039410 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:28.039412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:11:28.039511 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:28.039513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.040148 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.040149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:28.040918 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:28.040920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.041587 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.041588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:28.041668 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:28.041673 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:11:28.041771 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 23:11:28.041814 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":3,"thread_start_us":153,"threads_started":1}
I20260620 23:11:28.042344 17303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5830027b4c84dc690defd1b3a2ec518" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" is_pre_election: true
I20260620 23:11:28.042516 17303 raft_consensus.cc:2454] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 0.
I20260620 23:11:28.042449 17435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5830027b4c84dc690defd1b3a2ec518" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" is_pre_election: true
I20260620 23:11:28.042639 17435 raft_consensus.cc:2454] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 0.
I20260620 23:11:28.042737 17097 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 733022fef359484abb7eefa719c683cd; no voters:
I20260620 23:11:28.042868 17217 raft_consensus.cc:2781] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:11:28.042943 17217 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:11:28.042969 17217 raft_consensus.cc:3037] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:11:28.043663 17217 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:28.043814 17217 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 election: Requested vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
I20260620 23:11:28.043987 17303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5830027b4c84dc690defd1b3a2ec518" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59"
I20260620 23:11:28.044034 17435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5830027b4c84dc690defd1b3a2ec518" candidate_uuid: "733022fef359484abb7eefa719c683cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8664b12b2b7b4d34b41cc4cf6088a484"
I20260620 23:11:28.044076 17303 raft_consensus.cc:3037] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:11:28.044150 17435 raft_consensus.cc:3037] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:11:28.044812 17303 raft_consensus.cc:2454] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 1.
I20260620 23:11:28.044950 17435 raft_consensus.cc:2454] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 733022fef359484abb7eefa719c683cd in term 1.
I20260620 23:11:28.044973 17097 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 733022fef359484abb7eefa719c683cd; no voters:
I20260620 23:11:28.045058 17217 raft_consensus.cc:2781] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:11:28.045212 17217 raft_consensus.cc:686] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 LEADER]: Becoming Leader. State: Replica: 733022fef359484abb7eefa719c683cd, State: Running, Role: LEADER
I20260620 23:11:28.045264 17217 consensus_queue.cc:237] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:28.046002 17023 catalog_manager.cc:5495] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 733022fef359484abb7eefa719c683cd (127.1.32.65). New cstate: current_term: 1 leader_uuid: "733022fef359484abb7eefa719c683cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } health_report { overall_health: HEALTHY } } }
I20260620 23:11:28.051906 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.049891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36160 (local address 127.1.32.65:36629)
0620 23:11:28.049927 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:28.049928 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:28.049960 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:11:28.050025 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:28.050026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:28.050057 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:28.050109 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:11:28.050200 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.050626 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.051374 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.051565 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.051613 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:28.051735 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:11:28.051773 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 23:11:28.051868 (+ 95us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36160: token expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:11:28.051939 17219 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36160: token expired
I20260620 23:11:28.057662 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.052358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59904 (local address 127.1.32.126:38725)
0620 23:11:28.052417 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:28.052420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:28.052429 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:11:28.052462 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:28.052462 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:28.052469 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:28.052507 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:28.052955 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.053398 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.054069 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.054221 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.054263 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:28.055423 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:28.055424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:28.055424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:28.055430 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:28.056669 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:28.056973 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:28.056974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:28.056974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:28.057025 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:28.057233 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:28.057234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:28.057234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:28.057329 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:28.057402 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:28.057489 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 23:11:28.057555 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:11:28.060684 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.058500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.1.32.65:36629)
0620 23:11:28.058563 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:28.058565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:28.058573 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:11:28.058618 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:28.058619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:28.058628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:28.058680 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:11:28.058805 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.059312 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.059955 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:28.060163 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:28.060237 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:28.060376 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:11:28.060479 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:11:28.060547 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:28.060602 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 23:11:28.060616 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:11:28.061968 17435 raft_consensus.cc:1261] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Refusing update from remote peer 733022fef359484abb7eefa719c683cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:11:28.061976 17303 raft_consensus.cc:1261] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Refusing update from remote peer 733022fef359484abb7eefa719c683cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:11:28.062309 17217 consensus_queue.cc:1035] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:11:28.062451 17498 consensus_queue.cc:1035] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:11:28.064702 17500 mvcc.cc:205] Tried to move back new op lower bound from 7299060072699736064 to 7299060072633987072. Current Snapshot: MvccSnapshot[applied={T|T < 7299060072699736064}]
I20260620 23:11:28.064810 17502 mvcc.cc:205] Tried to move back new op lower bound from 7299060072699736064 to 7299060072633987072. Current Snapshot: MvccSnapshot[applied={T|T < 7299060072699736064}]
I20260620 23:11:28.066772 17023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:11:28.066869 17023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:11:28.067574 17023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Sending DeleteTablet for 3 replicas of tablet d5830027b4c84dc690defd1b3a2ec518
I20260620 23:11:28.067869 17145 tablet_service.cc:1514] Processing DeleteTablet for tablet d5830027b4c84dc690defd1b3a2ec518 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:11:28 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:36152
I20260620 23:11:28.067884 17283 tablet_service.cc:1514] Processing DeleteTablet for tablet d5830027b4c84dc690defd1b3a2ec518 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:11:28 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:58452
I20260620 23:11:28.067902 17415 tablet_service.cc:1514] Processing DeleteTablet for tablet d5830027b4c84dc690defd1b3a2ec518 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:11:28 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:50986
I20260620 23:11:28.067948 17220 tablet_replica.cc:330] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: stopping tablet replica
I20260620 23:11:28.067997 17220 raft_consensus.cc:2229] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:11:28.068105 17220 raft_consensus.cc:2258] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:11:28.068136 17511 tablet_replica.cc:330] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: stopping tablet replica
I20260620 23:11:28.068140 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 17061
I20260620 23:11:28.069320 17510 tablet_replica.cc:330] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: stopping tablet replica
I20260620 23:11:28.069454 17510 raft_consensus.cc:2229] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:11:28.069592 17510 raft_consensus.cc:2258] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:11:28.070034 17510 ts_tablet_manager.cc:1905] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:11:28.070971 17511 raft_consensus.cc:2229] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:11:28.071110 17511 raft_consensus.cc:2258] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:11:28.071585 17511 ts_tablet_manager.cc:1905] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:11:28.071982 17510 ts_tablet_manager.cc:1918] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:11:28.072057 17510 log.cc:1196] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/d5830027b4c84dc690defd1b3a2ec518
I20260620 23:11:28.072333 17510 ts_tablet_manager.cc:1939] T d5830027b4c84dc690defd1b3a2ec518 P 487f0c4fd6b24fbc8cc61cf2e00ddd59: Deleting consensus metadata
I20260620 23:11:28.072783 17007 catalog_manager.cc:4841] TS 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): tablet d5830027b4c84dc690defd1b3a2ec518 (table test-table [id=4221ffa49eaf43ecb0ca9c0c7f9f6bd8]) successfully deleted
I20260620 23:11:28.072985 17511 ts_tablet_manager.cc:1918] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:11:28.073072 17511 log.cc:1196] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/d5830027b4c84dc690defd1b3a2ec518
I20260620 23:11:28.073311 17511 ts_tablet_manager.cc:1939] T d5830027b4c84dc690defd1b3a2ec518 P 8664b12b2b7b4d34b41cc4cf6088a484: Deleting consensus metadata
I20260620 23:11:28.073704 17010 catalog_manager.cc:4841] TS 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): tablet d5830027b4c84dc690defd1b3a2ec518 (table test-table [id=4221ffa49eaf43ecb0ca9c0c7f9f6bd8]) successfully deleted
W20260620 23:11:28.074553 17010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:36629 (error 108)
I20260620 23:11:28.075037 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:28.074946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:36629 (local address 127.0.0.1:36174)
0620 23:11:28.074978 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:28.075011 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260620 23:11:28.075145 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 17216
W20260620 23:11:28.075193 17010 catalog_manager.cc:4584] TS 733022fef359484abb7eefa719c683cd (127.1.32.65:36629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5830027b4c84dc690defd1b3a2ec518: Network error: Client connection negotiation failed: client connection to 127.1.32.65:36629: connect: Connection refused (error 111)
I20260620 23:11:28.080677 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 17352
I20260620 23:11:28.085114 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 16990
I20260620 23:11:48.090564 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38725
--webserver_interface=127.1.32.126
--webserver_port=39261
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:38725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:48.167585 17513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:48.167809 17513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:48.167850 17513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:48.168999 17513 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:11:48.169068 17513 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:11:48.169094 17513 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:11:48.169121 17513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:11:48.169167 17513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:48.169203 17513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:48.169234 17513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:11:48.169265 17513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:11:48.170820 17513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:38725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:38725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:48.171118 17513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:48.171344 17513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:48.176072 17519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.176079 17521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.176092 17518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:48.176286 17513 server_base.cc:1034] running on GCE node
I20260620 23:11:48.176541 17513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:48.176833 17513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:48.177999 17513 hybrid_clock.cc:648] HybridClock initialized: now 1781997108177988 us; error 31 us; skew 500 ppm
Jun 20 23:11:48 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:48.180395 17513 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:11:48.180786 17513 webserver.cc:460] Webserver started at http://127.1.32.126:39261/ using document root <none> and password file <none>
I20260620 23:11:48.181007 17513 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:48.181083 17513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:48.182405 17513 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:48.183182 17528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.183396 17513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:48.183502 17513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "415ae5995ecb43b6bd3bdf556cfb8346"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "04817768fb6f2ab89559821cd490a9e4"
server_key_iv: "3ab0d6bfc11fb3e261054cc482bd9f82"
server_key_version: "encryptionkey@0"
I20260620 23:11:48.183836 17513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:48.199695 17513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:48.201465 17513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:48.201607 17513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:48.205477 17513 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:38725
I20260620 23:11:48.205508 17580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:38725 every 8 connection(s)
I20260620 23:11:48.205825 17513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:11:48.207283 17581 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:11:48.207921 17581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Bootstrap starting.
I20260620 23:11:48.215339 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 17513
I20260620 23:11:48.215658 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:36629
--local_ip_for_outbound_sockets=127.1.32.65
--tserver_master_addrs=127.1.32.126:38725
--webserver_port=40449
--webserver_interface=127.1.32.65
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:11:48.216606 17581 log.cc:824] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:48.219475 17581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:11:48.219796 17581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Bootstrap complete.
I20260620 23:11:48.223695 17581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:48.224047 17581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415ae5995ecb43b6bd3bdf556cfb8346, State: Initialized, Role: FOLLOWER
I20260620 23:11:48.224225 17581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:48.224308 17581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:11:48.224361 17581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:11:48.224428 17581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:11:48.225926 17581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:48.226037 17581 leader_election.cc:302] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 415ae5995ecb43b6bd3bdf556cfb8346; no voters:
I20260620 23:11:48.226195 17581 leader_election.cc:288] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 23:11:48.226264 17585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 23:11:48.226418 17585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [term 3 LEADER]: Becoming Leader. State: Replica: 415ae5995ecb43b6bd3bdf556cfb8346, State: Running, Role: LEADER
I20260620 23:11:48.226457 17581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:11:48.226503 17585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } }
I20260620 23:11:48.226847 17586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } } }
I20260620 23:11:48.226936 17586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: This master's current role is: LEADER
I20260620 23:11:48.227138 17587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 415ae5995ecb43b6bd3bdf556cfb8346. Latest consensus state: current_term: 3 leader_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415ae5995ecb43b6bd3bdf556cfb8346" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 38725 } } }
I20260620 23:11:48.227252 17593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:11:48.227269 17587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346 [sys.catalog]: This master's current role is: LEADER
I20260620 23:11:48.228175 17593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:11:48.228268 17593 catalog_manager.cc:1259] Loaded cluster ID: 6cba6cb26c9f4f9dacb995491d82457a
I20260620 23:11:48.228300 17593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:11:48.229485 17593 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:11:48.229652 17593 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Loaded TSK: 0
I20260620 23:11:48.229696 17593 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 415ae5995ecb43b6bd3bdf556cfb8346: Loaded TSK: 1
F20260620 23:11:48.229916 17593 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb769eef9ad google::LogMessage::Fail() at ??:0
@ 0x7fb769ef288a google::LogMessage::SendToLog() at ??:0
@ 0x7fb769eef4de google::LogMessage::Flush() at ??:0
@ 0x7fb769ef0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb76c65c104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb76c659d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb771e491cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb771e4948e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb771e49594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb771e0fc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb771e19ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb76bdbd0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb76bdb2954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb76cc166db start_thread at ??:0
@ 0x7fb7689ed61f clone at ??:0
W20260620 23:11:48.318553 17584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:48.318734 17584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:48.318756 17584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:48.319868 17584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:48.319914 17584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:48.319940 17584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:11:48.321300 17584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:36629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=40449
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:48.321523 17584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:48.321771 17584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:48.326577 17623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.326699 17622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.326583 17625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:48.326831 17584 server_base.cc:1034] running on GCE node
I20260620 23:11:48.327057 17584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:48.327304 17584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:48.328476 17584 hybrid_clock.cc:648] HybridClock initialized: now 1781997108328466 us; error 34 us; skew 500 ppm
Jun 20 23:11:48 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:48.331056 17584 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:11:48.331504 17584 webserver.cc:460] Webserver started at http://127.1.32.65:40449/ using document root <none> and password file <none>
I20260620 23:11:48.331733 17584 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:48.331800 17584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:48.333206 17584 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:11:48.333946 17632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.334225 17584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:11:48.334322 17584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "733022fef359484abb7eefa719c683cd"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "e5c32389189276d2a8389a5e1d650ebe"
server_key_iv: "c5b5e028cac8fdaecf924c9934b6a467"
server_key_version: "encryptionkey@0"
I20260620 23:11:48.334748 17584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:48.357673 17584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:48.360106 17584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:48.360255 17584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:48.360517 17584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:48.360987 17639 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 23:11:48.362488 17584 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:11:48.362562 17584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 23:11:48.362596 17584 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:11:48.363354 17584 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:11:48.363397 17584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 23:11:48.363449 17639 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Bootstrap starting.
I20260620 23:11:48.371390 17584 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:36629
I20260620 23:11:48.371459 17746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:36629 every 8 connection(s)
I20260620 23:11:48.371948 17584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:11:48.374676 17639 log.cc:824] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Log is configured to *not* fsync() on all Append() calls
I20260620 23:11:48.377259 17639 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:11:48.377601 17639 tablet_bootstrap.cc:492] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Bootstrap complete.
I20260620 23:11:48.378856 17639 ts_tablet_manager.cc:1397] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260620 23:11:48.380249 17639 raft_consensus.cc:348] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:48.380584 17639 raft_consensus.cc:729] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733022fef359484abb7eefa719c683cd, State: Initialized, Role: FOLLOWER
I20260620 23:11:48.380726 17639 consensus_queue.cc:260] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:48.380986 17639 ts_tablet_manager.cc:1428] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260620 23:11:48.381492 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 17584
I20260620 23:11:48.381758 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:41113
--local_ip_for_outbound_sockets=127.1.32.66
--tserver_master_addrs=127.1.32.126:38725
--webserver_port=39859
--webserver_interface=127.1.32.66
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:48.422947 17749 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:11:48.372902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:53083)
0620 23:11:48.373134 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.373186 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:48.373920 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:48.422707 (+ 48787us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
W20260620 23:11:48.423194 17747 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:38725 (0 consecutive failures): Network error: Failed to ping master at 127.1.32.126:38725: Client connection negotiation failed: client connection to 127.1.32.126:38725: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:11:48.423569 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.423427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:55215)
0620 23:11:48.423485 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.423538 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:11:48.423954 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.423846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:47581)
0620 23:11:48.423907 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.423936 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260620 23:11:48.424108 17747 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:11:48.460822 17752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:48.461031 17752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:48.461066 17752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:48.462183 17752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:48.462218 17752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:48.462244 17752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:11:48.463691 17752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:41113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=39859
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:48.463871 17752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:48.464049 17752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:11:48.468930 17757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.468888 17758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.468948 17760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:48.469137 17752 server_base.cc:1034] running on GCE node
I20260620 23:11:48.469338 17752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:48.469594 17752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:48.470753 17752 hybrid_clock.cc:648] HybridClock initialized: now 1781997108470732 us; error 53 us; skew 500 ppm
Jun 20 23:11:48 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:48.473136 17752 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:11:48.473598 17752 webserver.cc:460] Webserver started at http://127.1.32.66:39859/ using document root <none> and password file <none>
I20260620 23:11:48.473831 17752 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:48.473898 17752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:48.475322 17752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:48.475906 17767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.476095 17752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:11:48.476189 17752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59"
format_stamp: "Formatted at 2026-06-20 23:11:06 on dist-test-slave-fzjm"
server_key: "b806c891c220059c23c0616fd8031042"
server_key_iv: "07522c983af1be5aeec7008f56f79417"
server_key_version: "encryptionkey@0"
I20260620 23:11:48.476547 17752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:48.484171 17752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:48.485952 17752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:48.486101 17752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:48.486303 17752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:48.486583 17752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:11:48.486634 17752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.486672 17752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:11:48.486698 17752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.492563 17752 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:41113
I20260620 23:11:48.492609 17880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:41113 every 8 connection(s)
I20260620 23:11:48.492911 17752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:11:48.493827 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.493495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:42977)
0620 23:11:48.493635 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.493698 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
W20260620 23:11:48.494186 17881 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:38725 (0 consecutive failures): Network error: Failed to ping master at 127.1.32.126:38725: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
I20260620 23:11:48.494416 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.494357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:40849)
0620 23:11:48.494382 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.494403 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
I20260620 23:11:48.494621 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.494575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:47815)
0620 23:11:48.494594 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.494611 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260620 23:11:48.494735 17881 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:11:48.496510 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 17752
I20260620 23:11:48.496776 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:39197
--local_ip_for_outbound_sockets=127.1.32.67
--tserver_master_addrs=127.1.32.126:38725
--webserver_port=39885
--webserver_interface=127.1.32.67
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:11:48.578724 17884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:11:48.579025 17884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:11:48.579052 17884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:11:48.580264 17884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:11:48.580334 17884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:11:48.580363 17884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:11:48.581789 17884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:39197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=39885
--tserver_master_addrs=127.1.32.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:11:48.581979 17884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:11:48.582155 17884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:11:48.587183 17884 server_base.cc:1034] running on GCE node
W20260620 23:11:48.587109 17892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.587141 17889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:11:48.587126 17890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:11:48.587540 17884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:11:48.587806 17884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:11:48.588968 17884 hybrid_clock.cc:648] HybridClock initialized: now 1781997108588957 us; error 31 us; skew 500 ppm
Jun 20 23:11:48 dist-test-slave-fzjm krb5kdc[16445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:11:48.591451 17884 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:11:48.591966 17884 webserver.cc:460] Webserver started at http://127.1.32.67:39885/ using document root <none> and password file <none>
I20260620 23:11:48.592201 17884 fs_manager.cc:359] Metadata directory not provided
I20260620 23:11:48.592245 17884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:11:48.593506 17884 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:11:48.594072 17899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.594238 17884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.594307 17884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "8664b12b2b7b4d34b41cc4cf6088a484"
format_stamp: "Formatted at 2026-06-20 23:11:07 on dist-test-slave-fzjm"
server_key: "26de81c8a6f31af62b0fe3b6f0a8c757"
server_key_iv: "dfcd6f6057b36fa28d0ee37b9b70aad5"
server_key_version: "encryptionkey@0"
I20260620 23:11:48.594647 17884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:11:48.608531 17884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:11:48.610790 17884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:11:48.610989 17884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:11:48.611241 17884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:11:48.611550 17884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:11:48.611595 17884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.611617 17884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:11:48.611689 17884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:11:48.617571 17884 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:39197
I20260620 23:11:48.617628 18012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:39197 every 8 connection(s)
I20260620 23:11:48.617954 17884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:11:48.619086 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.618620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:43289)
0620 23:11:48.618763 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.618829 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
W20260620 23:11:48.619493 18013 heartbeater.cc:644] Failed to heartbeat to 127.1.32.126:38725 (0 consecutive failures): Network error: Failed to ping master at 127.1.32.126:38725: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
I20260620 23:11:48.619750 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.619662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:48909)
0620 23:11:48.619707 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.619733 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 23:11:48.620021 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:48.619948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:42515)
0620 23:11:48.619986 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:48.620008 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 23:11:48.620131 18013 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:11:48.621636 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 17884
I20260620 23:11:49.424686 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:49.424460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:48215)
0620 23:11:49.424589 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:49.424628 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260620 23:11:49.495299 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:49.495095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:51073)
0620 23:11:49.495214 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:49.495248 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260620 23:11:49.620643 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:49.620433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:54533)
0620 23:11:49.620568 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:49.620599 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260620 23:11:50.062191 18020 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:11:50.062331 18020 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:50.062685 18020 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
Jun 20 23:11:50 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.66@KRBTEST.COM
Jun 20 23:11:50 dist-test-slave-fzjm krb5kdc[16445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.67@KRBTEST.COM
I20260620 23:11:50.073313 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:50.063049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:60123 (local address 127.1.32.67:39197)
0620 23:11:50.063203 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:50.063214 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:50.063242 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:11:50.063661 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:50.063663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:50.063704 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:50.063841 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:50.064567 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.065341 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.066317 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.066669 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.066793 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:50.069544 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:50.069548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:50.069548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:50.069557 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:50.071757 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:50.072076 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:50.072078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:50.072078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:50.072173 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:50.072366 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:50.072367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:50.072367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:50.072656 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:50.072736 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:50.073081 (+ 345us) server_negotiation.cc:299] Negotiation successful
0620 23:11:50.073204 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260620 23:11:50.073313 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:50.063018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:47597 (local address 127.1.32.66:41113)
0620 23:11:50.063214 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 23:11:50.063223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:11:50.063242 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:11:50.063666 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:11:50.063668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:11:50.063704 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:11:50.063841 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:11:50.064573 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.065340 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.066305 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.066667 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.066792 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:50.069167 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:11:50.069171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:11:50.069171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:11:50.069182 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:11:50.071501 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:50.071985 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:50.071988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:50.071988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:50.072112 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:11:50.072366 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:11:50.072367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:11:50.072367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:11:50.072656 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:11:50.072736 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:11:50.073081 (+ 345us) server_negotiation.cc:299] Negotiation successful
0620 23:11:50.073204 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":64,"threads_started":1}
I20260620 23:11:50.073341 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:50.062855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:41113 (local address 127.1.32.65:47597)
0620 23:11:50.063016 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:50.063027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:50.063085 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:50.063885 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:50.063888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:50.064251 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:50.064463 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:50.064468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.065433 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.065434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:50.066222 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:50.066224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.066764 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.066765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:50.066878 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:50.067518 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:50.067528 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:50.069066 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:50.071601 (+ 2535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:50.071607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:50.071609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:50.071864 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:50.072174 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:50.072174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:50.072175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:50.072304 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:50.072745 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:50.072754 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:50.072924 (+ 170us) client_negotiation.cc:769] Sending connection context
0620 23:11:50.073023 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 23:11:50.073206 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":67,"thread_start_us":94,"threads_started":1}
I20260620 23:11:50.073626 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:50.062855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:39197 (local address 127.1.32.65:60123)
0620 23:11:50.063038 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:50.063050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:11:50.063115 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:11:50.063893 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:11:50.063894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:11:50.064253 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:11:50.064463 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:50.064468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.065448 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.065449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:50.066223 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:11:50.066226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:11:50.066760 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:11:50.066761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:11:50.066877 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:11:50.067529 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:11:50.067534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:11:50.069422 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:11:50.071830 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:50.071831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:50.071831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:50.072017 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:50.072228 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:11:50.072229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:11:50.072229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:11:50.072304 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:11:50.072745 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:11:50.072754 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:11:50.072908 (+ 154us) client_negotiation.cc:769] Sending connection context
0620 23:11:50.073023 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 23:11:50.073256 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":182,"thread_start_us":82,"threads_started":1}
W20260620 23:11:50.074414 17633 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
W20260620 23:11:50.074527 17636 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
I20260620 23:11:50.074594 17636 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 733022fef359484abb7eefa719c683cd; no voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 8664b12b2b7b4d34b41cc4cf6088a484
I20260620 23:11:50.074757 18020 raft_consensus.cc:2726] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:11:50.425257 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:50.425135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:59793)
0620 23:11:50.425188 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:50.425219 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:11:50.495927 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:50.495707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:59649)
0620 23:11:50.495827 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:50.495876 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260620 23:11:50.621272 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:50.621040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:45875)
0620 23:11:50.621196 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:50.621233 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260620 23:11:51.425938 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:51.425704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:37623)
0620 23:11:51.425840 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:51.425874 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260620 23:11:51.496552 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:51.496382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:39075)
0620 23:11:51.496486 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:51.496516 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
I20260620 23:11:51.621866 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:51.621662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:57845)
0620 23:11:51.621798 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:51.621832 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260620 23:11:51.997438 18030 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:11:51.997543 18030 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:51.997920 18030 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
W20260620 23:11:51.998121 17633 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
W20260620 23:11:51.998190 17636 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
I20260620 23:11:51.998257 17636 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 733022fef359484abb7eefa719c683cd; no voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 8664b12b2b7b4d34b41cc4cf6088a484
I20260620 23:11:51.998387 18030 raft_consensus.cc:2726] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:11:52.426604 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:52.426399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:37005)
0620 23:11:52.426520 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:52.426557 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260620 23:11:52.497227 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:52.497004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:39279)
0620 23:11:52.497115 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:52.497168 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260620 23:11:52.622582 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:52.622310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:55441)
0620 23:11:52.622482 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:52.622518 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260620 23:11:53.427320 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:53.427092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:38525)
0620 23:11:53.427239 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:53.427276 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":62,"threads_started":1}
I20260620 23:11:53.497889 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:53.497691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:50449)
0620 23:11:53.497810 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:53.497846 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260620 23:11:53.623320 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:53.623130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:48427)
0620 23:11:53.623245 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:53.623280 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260620 23:11:54.104367 18037 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:11:54.104463 18037 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:54.104672 18037 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
W20260620 23:11:54.105027 17636 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
W20260620 23:11:54.105106 17633 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
I20260620 23:11:54.105170 17636 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 733022fef359484abb7eefa719c683cd; no voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 8664b12b2b7b4d34b41cc4cf6088a484
I20260620 23:11:54.105261 18037 raft_consensus.cc:2726] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:11:54.427990 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:54.427730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:55811)
0620 23:11:54.427888 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:54.427943 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":55,"threads_started":1}
I20260620 23:11:54.498617 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:54.498345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:37961)
0620 23:11:54.498535 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:54.498576 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":68,"threads_started":1}
I20260620 23:11:54.624029 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:54.623856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:47681)
0620 23:11:54.623962 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:54.623993 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260620 23:11:55.428686 18041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:55.428442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:55307)
0620 23:11:55.428591 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:55.428624 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260620 23:11:55.499174 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:55.498989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:56999)
0620 23:11:55.499103 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:55.499136 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260620 23:11:55.624686 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:55.624482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:34021)
0620 23:11:55.624607 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:55.624640 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260620 23:11:55.873556 18044 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:11:55.873672 18044 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:11:55.873883 18044 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
W20260620 23:11:55.874254 17633 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
W20260620 23:11:55.874313 17636 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
I20260620 23:11:55.874380 17636 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 733022fef359484abb7eefa719c683cd; no voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 8664b12b2b7b4d34b41cc4cf6088a484
I20260620 23:11:55.874470 18044 raft_consensus.cc:2726] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:11:56.429411 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:56.429196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:53709)
0620 23:11:56.429326 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:56.429358 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260620 23:11:56.499866 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:56.499645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:36699)
0620 23:11:56.499781 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:56.499821 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260620 23:11:56.625322 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:56.625129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:37031)
0620 23:11:56.625248 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:56.625284 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260620 23:11:57.429984 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:57.429793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:36765)
0620 23:11:57.429908 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:57.429938 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260620 23:11:57.500530 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:57.500335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:36749)
0620 23:11:57.500460 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:57.500492 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260620 23:11:57.625957 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:57.625761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:46061)
0620 23:11:57.625893 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:57.625921 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":53,"threads_started":1}
I20260620 23:11:58.430635 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:58.430414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:58141)
0620 23:11:58.430549 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:58.430587 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260620 23:11:58.501137 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:58.500919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:55917)
0620 23:11:58.501061 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:58.501097 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260620 23:11:58.626540 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:58.626339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:55687)
0620 23:11:58.626468 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:58.626501 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260620 23:11:59.431396 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:59.431201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:46929)
0620 23:11:59.431316 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:59.431351 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260620 23:11:59.501905 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:59.501637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:41293)
0620 23:11:59.501794 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:59.501829 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260620 23:11:59.627274 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:11:59.627056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:39581)
0620 23:11:59.627186 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 23:11:59.627227 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260620 23:12:00.432013 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:00.431795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:45727)
0620 23:12:00.431933 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:00.431968 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260620 23:12:00.502522 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:00.502331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:50761)
0620 23:12:00.502453 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:00.502487 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":55,"threads_started":1}
I20260620 23:12:00.627887 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:00.627704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:56893)
0620 23:12:00.627816 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:00.627847 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260620 23:12:01.065356 18060 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:01.065451 18060 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:12:01.065614 18060 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
W20260620 23:12:01.066061 17633 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
W20260620 23:12:01.066185 17636 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
I20260620 23:12:01.066263 17636 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 733022fef359484abb7eefa719c683cd; no voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 8664b12b2b7b4d34b41cc4cf6088a484
I20260620 23:12:01.066361 18060 raft_consensus.cc:2726] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:12:01.432715 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:01.432490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:41007)
0620 23:12:01.432624 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:01.432671 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260620 23:12:01.503232 18062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:01.503039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:56329)
0620 23:12:01.503155 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:01.503190 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
I20260620 23:12:01.628624 18063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:01.628416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:42761)
0620 23:12:01.628551 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:01.628585 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260620 23:12:02.433365 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:02.433177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:51403)
0620 23:12:02.433284 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:02.433317 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260620 23:12:02.503991 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:02.503668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:45579)
0620 23:12:02.503856 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:02.503891 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
I20260620 23:12:02.629280 18066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:02.629068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:53227)
0620 23:12:02.629209 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:02.629241 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260620 23:12:03.434093 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:03.433904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.65:34329)
0620 23:12:03.434021 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:03.434052 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260620 23:12:03.504631 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:03.504441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.66:55669)
0620 23:12:03.504563 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:03.504593 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260620 23:12:03.606369 18069 raft_consensus.cc:482] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:03.606474 18069 raft_consensus.cc:504] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "487f0c4fd6b24fbc8cc61cf2e00ddd59" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 41113 } } peers { permanent_uuid: "8664b12b2b7b4d34b41cc4cf6088a484" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 39197 } } peers { permanent_uuid: "733022fef359484abb7eefa719c683cd" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 36629 } }
I20260620 23:12:03.606669 18069 leader_election.cc:288] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113), 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197)
W20260620 23:12:03.607052 17633 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 487f0c4fd6b24fbc8cc61cf2e00ddd59 (127.1.32.66:41113): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
W20260620 23:12:03.607116 17636 leader_election.cc:341] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8664b12b2b7b4d34b41cc4cf6088a484 (127.1.32.67:39197): Not found: Tablet not found: d5830027b4c84dc690defd1b3a2ec518
I20260620 23:12:03.607170 17636 leader_election.cc:302] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 733022fef359484abb7eefa719c683cd; no voters: 487f0c4fd6b24fbc8cc61cf2e00ddd59, 8664b12b2b7b4d34b41cc4cf6088a484
I20260620 23:12:03.607247 18069 raft_consensus.cc:2726] T d5830027b4c84dc690defd1b3a2ec518 P 733022fef359484abb7eefa719c683cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:12:03.628022 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 17584
I20260620 23:12:03.629982 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:03.629795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:38725 (local address 127.1.32.67:46847)
0620 23:12:03.629902 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:03.629935 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.126:38725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260620 23:12:03.632776 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 17752
I20260620 23:12:03.636912 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 17884
2026-06-20T23:12:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58204 ms)
[----------] 30 tests from SecurityITest (225741 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:03 dist-test-slave-fzjm krb5kdc[18073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:03 dist-test-slave-fzjm krb5kdc[18073](info): set up 2 sockets
Jun 20 23:12:03 dist-test-slave-fzjm krb5kdc[18073](info): commencing operation
krb5kdc: starting...
W20260620 23:12:04.642357 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:04 dist-test-slave-fzjm krb5kdc[18073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:04.759011 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39403
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:39403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:04.837873 18086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:04.838064 18086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:04.838088 18086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:04.839196 18086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:04.839241 18086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:04.839255 18086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:04.839267 18086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:04.839278 18086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:04.840647 18086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:39403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:04.840826 18086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:04.841022 18086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:04.845685 18092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:04.845671 18094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:04.845963 18086 server_base.cc:1034] running on GCE node
W20260620 23:12:04.845657 18091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:04.846266 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:04.846519 18086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:04.847690 18086 hybrid_clock.cc:648] HybridClock initialized: now 1781997124847671 us; error 29 us; skew 500 ppm
Jun 20 23:12:04 dist-test-slave-fzjm krb5kdc[18073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997124, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:04.849928 18086 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:04.850359 18086 webserver.cc:460] Webserver started at http://127.1.32.126:38129/ using document root <none> and password file <none>
I20260620 23:12:04.850584 18086 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:04.850652 18086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:04.850809 18086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:04.851862 18086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "8214e7e5e9ed4562bc9e32c33b39a87e"
format_stamp: "Formatted at 2026-06-20 23:12:04 on dist-test-slave-fzjm"
server_key: "e208ddf8a8acc8ee3304cc942f1e840a"
server_key_iv: "4107fa485f5a92b5bbc43e3b5758094b"
server_key_version: "encryptionkey@0"
I20260620 23:12:04.852223 18086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "8214e7e5e9ed4562bc9e32c33b39a87e"
format_stamp: "Formatted at 2026-06-20 23:12:04 on dist-test-slave-fzjm"
server_key: "e208ddf8a8acc8ee3304cc942f1e840a"
server_key_iv: "4107fa485f5a92b5bbc43e3b5758094b"
server_key_version: "encryptionkey@0"
I20260620 23:12:04.853505 18086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:04.854261 18101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:04.854415 18086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:04.854506 18086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "8214e7e5e9ed4562bc9e32c33b39a87e"
format_stamp: "Formatted at 2026-06-20 23:12:04 on dist-test-slave-fzjm"
server_key: "e208ddf8a8acc8ee3304cc942f1e840a"
server_key_iv: "4107fa485f5a92b5bbc43e3b5758094b"
server_key_version: "encryptionkey@0"
I20260620 23:12:04.854609 18086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:04.864311 18086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:04.866073 18086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:04.866226 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:04.870218 18086 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:39403
I20260620 23:12:04.870271 18153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:39403 every 8 connection(s)
I20260620 23:12:04.870636 18086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:04.871521 18154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:04.873399 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18086
I20260620 23:12:04.873518 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:04.873708 1153 external_mini_cluster.cc:1428] Setting key c822f7d28286e2c4192ee6be0534ae20
I20260620 23:12:04.873795 18154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e: Bootstrap starting.
I20260620 23:12:04.874527 18154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:04.874835 18154 log.cc:824] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:04.875509 18154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e: No bootstrap required, opened a new log
I20260620 23:12:04.877208 18154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39403 } }
I20260620 23:12:04.877351 18154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:04.877396 18154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8214e7e5e9ed4562bc9e32c33b39a87e, State: Initialized, Role: FOLLOWER
I20260620 23:12:04.877511 18154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39403 } }
I20260620 23:12:04.877600 18154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:04.877642 18154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:04.877687 18154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:04.878191 18154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39403 } }
I20260620 23:12:04.878300 18154 leader_election.cc:302] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8214e7e5e9ed4562bc9e32c33b39a87e; no voters:
I20260620 23:12:04.878463 18154 leader_election.cc:288] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:04.878556 18159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:04.878705 18159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [term 1 LEADER]: Becoming Leader. State: Replica: 8214e7e5e9ed4562bc9e32c33b39a87e, State: Running, Role: LEADER
I20260620 23:12:04.878773 18154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:04.878856 18159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39403 } }
I20260620 23:12:04.879294 18160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39403 } } }
I20260620 23:12:04.879384 18160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:04.879617 18161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8214e7e5e9ed4562bc9e32c33b39a87e. Latest consensus state: current_term: 1 leader_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8214e7e5e9ed4562bc9e32c33b39a87e" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39403 } } }
I20260620 23:12:04.879734 18161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:04.879900 18167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 20 23:12:04 dist-test-slave-fzjm krb5kdc[18073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:04.880362 18167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:04.881961 18167 catalog_manager.cc:1347] Generated new cluster ID: 882286c627bd4c72b873c21abf18f9d8
I20260620 23:12:04.882014 18167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:04.884084 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:04.874216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.1.32.126:39403)
0620 23:12:04.874386 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:04.874400 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:04.874427 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:12:04.874854 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:04.874862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:04.874962 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:04.875101 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:04.875724 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:04.876571 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:04.877317 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:04.877862 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:04.877987 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:04.880805 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:04.880810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:04.880810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:04.880822 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:04.882617 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:04.882986 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:04.882990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:04.882990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:04.883073 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:04.883294 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:04.883295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:04.883295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:04.883515 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:04.883679 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:04.883748 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:12:04.883893 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260620 23:12:04.896138 18167 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:04.896629 18167 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:04.908721 18167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e: Generated new TSK 0
I20260620 23:12:04.908903 18167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:12:04.938676 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39403
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:05.017650 18180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:05.017849 18180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:05.017872 18180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:05.018985 18180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:05.019024 18180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:05.019049 18180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:12:05.020489 18180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:05.020730 18180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:05.020969 18180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:05.025648 18186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.025713 18188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.025709 18185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.025930 18180 server_base.cc:1034] running on GCE node
I20260620 23:12:05.026152 18180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:05.026382 18180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:05.027536 18180 hybrid_clock.cc:648] HybridClock initialized: now 1781997125027530 us; error 36 us; skew 500 ppm
Jun 20 23:12:05 dist-test-slave-fzjm krb5kdc[18073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997125, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:05.030020 18180 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:12:05.030417 18180 webserver.cc:460] Webserver started at http://127.1.32.65:41773/ using document root <none> and password file <none>
I20260620 23:12:05.030644 18180 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:05.030717 18180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:05.030864 18180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:05.031860 18180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "5c1d4c1c2db447a5b4fd5faa633927ca"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "747bff7b319bfa9b9c62148cf6dda64e"
server_key_iv: "687f374f7abd549792e7aae506219f97"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.032256 18180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "5c1d4c1c2db447a5b4fd5faa633927ca"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "747bff7b319bfa9b9c62148cf6dda64e"
server_key_iv: "687f374f7abd549792e7aae506219f97"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.033566 18180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:12:05.034314 18195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.034512 18180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:05.034600 18180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "5c1d4c1c2db447a5b4fd5faa633927ca"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "747bff7b319bfa9b9c62148cf6dda64e"
server_key_iv: "687f374f7abd549792e7aae506219f97"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.034678 18180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:05.072642 18180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:05.075136 18180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:05.075309 18180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:05.075567 18180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:05.075903 18180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:05.075960 18180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.076004 18180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:05.076082 18180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.082374 18180 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:42227
I20260620 23:12:05.082414 18308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:42227 every 8 connection(s)
I20260620 23:12:05.082768 18180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:12:05.083377 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18180
I20260620 23:12:05.083480 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:12:05.083684 1153 external_mini_cluster.cc:1428] Setting key 5e51d5511bb1d0b1b6483ea6dcf78c64
Jun 20 23:12:05 dist-test-slave-fzjm krb5kdc[18073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997125, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:12:05.092779 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.083586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:51959 (local address 127.1.32.126:39403)
0620 23:12:05.083642 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.083644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.083895 (+ 251us) server_negotiation.cc:407] Connection header received
0620 23:12:05.084559 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.084560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.084573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.084646 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:05.085619 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.086141 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.087334 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.087530 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.087606 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.089912 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:05.089914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:05.089914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:05.089922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:05.091404 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.091797 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.091798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.091799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.091863 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.092125 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.092126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.092126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.092237 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:05.092302 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.092561 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.092656 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:12:05.092947 18311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.083585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39403 (local address 127.1.32.65:51959)
0620 23:12:05.083775 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.083817 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.084459 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.084703 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.084706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.085265 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:05.085491 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.085498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.086236 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.086237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.087257 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.087260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.087614 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.087615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.087742 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.088229 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:05.088237 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:05.089803 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:05.091494 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:05.091501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:05.091503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:05.091716 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:05.091942 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:05.091943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:05.091943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:05.092055 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:05.092306 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:05.092313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:05.092438 (+ 125us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.092542 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.092684 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260620 23:12:05.093463 18309 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39403
I20260620 23:12:05.093569 18309 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:05.093761 18309 heartbeater.cc:511] Master 127.1.32.126:39403 requested a full tablet report, sending...
I20260620 23:12:05.094273 18118 ts_manager.cc:194] Registered new tserver with Master: 5c1d4c1c2db447a5b4fd5faa633927ca (127.1.32.65:42227)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
I20260620 23:12:05.095028 18118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:51959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:12:05.109433 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39403
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:05.186281 18314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:05.186508 18314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:05.186543 18314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:05.187599 18314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:05.187667 18314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:05.187705 18314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:12:05.188989 18314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:05.189227 18314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:05.189422 18314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:05.193979 18322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.194026 18320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.194159 18314 server_base.cc:1034] running on GCE node
W20260620 23:12:05.194031 18319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.194415 18314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:05.194610 18314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:05.195739 18314 hybrid_clock.cc:648] HybridClock initialized: now 1781997125195726 us; error 27 us; skew 500 ppm
Jun 20 23:12:05 dist-test-slave-fzjm krb5kdc[18073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997125, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:05.198117 18314 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:12:05.198537 18314 webserver.cc:460] Webserver started at http://127.1.32.66:40733/ using document root <none> and password file <none>
I20260620 23:12:05.198767 18314 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:05.198844 18314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:05.199045 18314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:05.200047 18314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "026ce0c39c944d6a9ca6db536c39d529"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "d86fb1f6b92eefd0ef9a329dd759af39"
server_key_iv: "3f3a6f405281fae3732b2c44c16cd640"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.200423 18314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "026ce0c39c944d6a9ca6db536c39d529"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "d86fb1f6b92eefd0ef9a329dd759af39"
server_key_iv: "3f3a6f405281fae3732b2c44c16cd640"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.201705 18314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:05.202397 18329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.202574 18314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:05.202697 18314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "026ce0c39c944d6a9ca6db536c39d529"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "d86fb1f6b92eefd0ef9a329dd759af39"
server_key_iv: "3f3a6f405281fae3732b2c44c16cd640"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.202781 18314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:05.215981 18314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:05.217787 18314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:05.217948 18314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:05.218155 18314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:05.218461 18314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:05.218518 18314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.218559 18314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:05.218595 18314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.225004 18314 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:40507
I20260620 23:12:05.225040 18442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:40507 every 8 connection(s)
I20260620 23:12:05.225355 18314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:12:05 dist-test-slave-fzjm krb5kdc[18073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997125, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:05.233747 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18314
I20260620 23:12:05.233866 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:12:05.234150 1153 external_mini_cluster.cc:1428] Setting key f2459bdc9304c5fac5b018b7fd738513
I20260620 23:12:05.234354 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.226082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:36875 (local address 127.1.32.126:39403)
0620 23:12:05.226150 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.226153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.226313 (+ 160us) server_negotiation.cc:407] Connection header received
0620 23:12:05.226727 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.226728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.226739 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.226799 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:05.227735 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.228214 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.229245 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.229435 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.229487 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.231695 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:05.231697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:05.231697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:05.231705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:05.232953 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.233328 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.233329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.233329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.233376 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.233605 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.233606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.233606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.233751 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:05.233827 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.234157 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.234245 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:12:05.234618 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.226031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39403 (local address 127.1.32.66:36875)
0620 23:12:05.226184 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.226220 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.226647 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.226867 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.226870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.227439 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:05.227630 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.227634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.228343 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.228344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.229162 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.229165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.229531 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.229533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.229640 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.230115 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:05.230122 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:05.231608 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:05.233029 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:05.233035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:05.233037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:05.233260 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:05.233442 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:05.233443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:05.233443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:05.233540 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:05.233846 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:05.233852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:05.234012 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.234103 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.234280 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260620 23:12:05.235189 18443 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39403
I20260620 23:12:05.235291 18443 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:05.235500 18443 heartbeater.cc:511] Master 127.1.32.126:39403 requested a full tablet report, sending...
I20260620 23:12:05.235909 18118 ts_manager.cc:194] Registered new tserver with Master: 026ce0c39c944d6a9ca6db536c39d529 (127.1.32.66:40507)
I20260620 23:12:05.236312 18118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:36875
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:12:05.257302 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39403
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:05.335038 18448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:05.335242 18448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:05.335263 18448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:05.336911 18448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:05.336967 18448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:05.337003 18448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:12:05.339200 18448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:05.339423 18448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:05.339656 18448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:05.345386 18454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.345408 18453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.345652 18456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.345701 18448 server_base.cc:1034] running on GCE node
I20260620 23:12:05.345942 18448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:05.346275 18448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:05.347651 18448 hybrid_clock.cc:648] HybridClock initialized: now 1781997125347628 us; error 38 us; skew 500 ppm
Jun 20 23:12:05 dist-test-slave-fzjm krb5kdc[18073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997125, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:05.350466 18448 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:12:05.350940 18448 webserver.cc:460] Webserver started at http://127.1.32.67:38873/ using document root <none> and password file <none>
I20260620 23:12:05.351161 18448 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:05.351219 18448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:05.351373 18448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:05.352322 18448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "6874189d51ee4e878da117069bfb2f68"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "9847aa67ed102451bd30fe2c7d314977"
server_key_iv: "888b7b714e556a6f3d53be49fe31a884"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.352679 18448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "6874189d51ee4e878da117069bfb2f68"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "9847aa67ed102451bd30fe2c7d314977"
server_key_iv: "888b7b714e556a6f3d53be49fe31a884"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.354223 18448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:05.355355 18463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.355533 18448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:05.355646 18448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "6874189d51ee4e878da117069bfb2f68"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "9847aa67ed102451bd30fe2c7d314977"
server_key_iv: "888b7b714e556a6f3d53be49fe31a884"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.355721 18448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:05.397831 18448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:05.400149 18448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:05.400250 18448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:05.400424 18448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:05.400686 18448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:05.400713 18448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.400734 18448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:05.400745 18448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.406518 18448 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:41691
I20260620 23:12:05.406603 18576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:41691 every 8 connection(s)
I20260620 23:12:05.406936 18448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:12:05.411927 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18448
I20260620 23:12:05.412035 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:12:05.412222 1153 external_mini_cluster.cc:1428] Setting key b26d804dc73a0e7b971ad406571b635d
Jun 20 23:12:05 dist-test-slave-fzjm krb5kdc[18073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997125, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:05.416109 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.407571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:42637 (local address 127.1.32.126:39403)
0620 23:12:05.407645 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.407647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.407837 (+ 190us) server_negotiation.cc:407] Connection header received
0620 23:12:05.408355 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.408356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.408368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.408438 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:05.409462 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.409976 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.410992 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.411197 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.411298 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.413561 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:05.413562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:05.413563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:05.413570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:05.414748 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.415196 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.415197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.415197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.415252 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.415496 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.415497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.415497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.415609 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:05.415681 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.415947 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.416023 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 23:12:05.416290 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.407565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39403 (local address 127.1.32.67:42637)
0620 23:12:05.407731 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.407762 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.408244 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.408510 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.408514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.409188 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:05.409365 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.409369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.410067 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.410068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.410851 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.410853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.411309 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.411310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.411436 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.411827 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:05.411835 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:05.413473 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:05.414822 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:05.414828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:05.414830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:05.415108 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:05.415323 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:05.415323 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:05.415324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:05.415430 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:05.415672 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:05.415677 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:05.415809 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.415925 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.416071 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260620 23:12:05.416744 18577 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39403
I20260620 23:12:05.416849 18577 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:05.417016 18577 heartbeater.cc:511] Master 127.1.32.126:39403 requested a full tablet report, sending...
I20260620 23:12:05.417367 18118 ts_manager.cc:194] Registered new tserver with Master: 6874189d51ee4e878da117069bfb2f68 (127.1.32.67:41691)
I20260620 23:12:05.417677 18118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:42637
I20260620 23:12:05.423740 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:12:05.431069 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.424958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43650 (local address 127.1.32.126:39403)
0620 23:12:05.425030 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.425033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.425128 (+ 95us) server_negotiation.cc:407] Connection header received
0620 23:12:05.425222 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.425223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.425232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.425296 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:05.425846 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.426301 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.427118 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.427299 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.427371 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.428496 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:05.428498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:05.428498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:05.428505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:05.429922 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.430273 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.430275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.430275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.430328 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:05.430520 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:05.430520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:05.430521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:05.430674 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:05.430775 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.430881 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.430974 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 23:12:05.432718 18118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:12:05.434260 18118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:12:05.440738 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.436477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:40507 (local address 127.0.0.1:50316)
0620 23:12:05.436921 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.436960 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.437030 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.437244 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.437247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.437249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:05.437312 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.437313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.438403 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.438404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.439596 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.439600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.440431 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.440432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.440518 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.440530 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.440599 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.440667 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":44,"threads_started":1}
I20260620 23:12:05.440748 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.436619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.1.32.66:40507)
0620 23:12:05.436859 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.436863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.437081 (+ 218us) server_negotiation.cc:407] Connection header received
0620 23:12:05.437127 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.437130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.437144 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.437240 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:05.437393 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.438295 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.439686 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.440331 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.440428 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.440501 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.440631 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.440670 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":77,"threads_started":1}
I20260620 23:12:05.441274 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.436553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42944 (local address 127.1.32.67:41691)
0620 23:12:05.436828 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.436834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.437188 (+ 354us) server_negotiation.cc:407] Connection header received
0620 23:12:05.437253 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.437255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.437269 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.437361 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:05.437598 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.438691 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.440057 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.440802 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.440892 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.440945 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.441144 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.441189 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260620 23:12:05.441416 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.436478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42227 (local address 127.0.0.1:34000)
0620 23:12:05.436920 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.436958 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.437044 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.437546 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.437547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.437549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:05.437602 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.437603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.438841 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.438842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.439927 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.439929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.441082 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.441084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.441156 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.441161 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.441222 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.441263 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":319,"thread_start_us":69,"threads_started":1}
I20260620 23:12:05.441497 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.436478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:41691 (local address 127.0.0.1:42944)
0620 23:12:05.437123 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.437130 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.437184 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.437458 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.437459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.437461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:05.437514 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.437516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.438816 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.438817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.439942 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.439944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.440978 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.440979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.441049 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.441054 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.441384 (+ 330us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.441428 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":352,"thread_start_us":75,"threads_started":1}
I20260620 23:12:05.441534 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.436614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.1.32.65:42227)
0620 23:12:05.436767 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.436773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.437041 (+ 268us) server_negotiation.cc:407] Connection header received
0620 23:12:05.437360 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.437363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.437379 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.437469 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:05.437684 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.438743 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.440025 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.440802 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.441075 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.441144 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.441236 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.441289 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260620 23:12:05.441720 18377 tablet_service.cc:1467] Processing CreateTablet for tablet 264deeff148a471ba472a1908b75f443 (DEFAULT_TABLE table=test-table [id=493eccbc20c8499fb32c78a88ee1adc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:05.442075 18377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264deeff148a471ba472a1908b75f443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:05.442183 18243 tablet_service.cc:1467] Processing CreateTablet for tablet 264deeff148a471ba472a1908b75f443 (DEFAULT_TABLE table=test-table [id=493eccbc20c8499fb32c78a88ee1adc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:05.442188 18511 tablet_service.cc:1467] Processing CreateTablet for tablet 264deeff148a471ba472a1908b75f443 (DEFAULT_TABLE table=test-table [id=493eccbc20c8499fb32c78a88ee1adc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:05.442487 18511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264deeff148a471ba472a1908b75f443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:05.442488 18243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264deeff148a471ba472a1908b75f443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:05.444741 18594 tablet_bootstrap.cc:492] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529: Bootstrap starting.
I20260620 23:12:05.445428 18594 tablet_bootstrap.cc:654] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:05.445763 18595 tablet_bootstrap.cc:492] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca: Bootstrap starting.
I20260620 23:12:05.446022 18596 tablet_bootstrap.cc:492] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Bootstrap starting.
I20260620 23:12:05.446038 18594 log.cc:824] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:05.446426 18595 tablet_bootstrap.cc:654] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:05.446681 18596 tablet_bootstrap.cc:654] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:05.446818 18594 tablet_bootstrap.cc:492] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529: No bootstrap required, opened a new log
I20260620 23:12:05.446949 18594 ts_tablet_manager.cc:1397] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:05.447029 18595 log.cc:824] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:05.447263 18596 log.cc:824] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:05.447849 18595 tablet_bootstrap.cc:492] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca: No bootstrap required, opened a new log
I20260620 23:12:05.447917 18595 ts_tablet_manager.cc:1397] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:05.447930 18596 tablet_bootstrap.cc:492] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: No bootstrap required, opened a new log
I20260620 23:12:05.447985 18596 ts_tablet_manager.cc:1397] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 23:12:05.449007 18594 raft_consensus.cc:348] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.449184 18594 raft_consensus.cc:374] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:05.449230 18594 raft_consensus.cc:729] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026ce0c39c944d6a9ca6db536c39d529, State: Initialized, Role: FOLLOWER
I20260620 23:12:05.449348 18594 consensus_queue.cc:260] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.449239 18595 raft_consensus.cc:348] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.449467 18595 raft_consensus.cc:374] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:05.449509 18595 raft_consensus.cc:729] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1d4c1c2db447a5b4fd5faa633927ca, State: Initialized, Role: FOLLOWER
I20260620 23:12:05.449611 18594 ts_tablet_manager.cc:1428] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 23:12:05.449621 18595 consensus_queue.cc:260] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.449738 18443 heartbeater.cc:503] Master 127.1.32.126:39403 was elected leader, sending a full tablet report...
I20260620 23:12:05.449877 18595 ts_tablet_manager.cc:1428] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:05.449952 18309 heartbeater.cc:503] Master 127.1.32.126:39403 was elected leader, sending a full tablet report...
I20260620 23:12:05.449950 18596 raft_consensus.cc:348] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.450107 18596 raft_consensus.cc:374] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:05.450147 18596 raft_consensus.cc:729] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6874189d51ee4e878da117069bfb2f68, State: Initialized, Role: FOLLOWER
I20260620 23:12:05.450253 18596 consensus_queue.cc:260] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.450517 18596 ts_tablet_manager.cc:1428] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 23:12:05.450537 18577 heartbeater.cc:503] Master 127.1.32.126:39403 was elected leader, sending a full tablet report...
W20260620 23:12:05.475878 18444 tablet.cc:2369] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:12:05.480885 18602 raft_consensus.cc:482] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:05.480971 18602 raft_consensus.cc:504] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.481242 18602 leader_election.cc:288] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 026ce0c39c944d6a9ca6db536c39d529 (127.1.32.66:40507), 5c1d4c1c2db447a5b4fd5faa633927ca (127.1.32.65:42227)
I20260620 23:12:05.484997 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.481477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:47195 (local address 127.1.32.66:40507)
0620 23:12:05.481520 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.481523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.481531 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:05.481620 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.481621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.481647 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.481698 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:05.481900 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.482615 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.484045 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.484620 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.484675 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.484683 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.484895 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.484934 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:12:05.485000 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.481508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:36473 (local address 127.1.32.65:42227)
0620 23:12:05.481546 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.481547 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.481677 (+ 130us) server_negotiation.cc:407] Connection header received
0620 23:12:05.481700 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.481701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.481711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.481758 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:05.481905 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.482615 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.484046 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.484594 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.484669 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.484680 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.484898 (+ 218us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.484939 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:05.485004 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.481409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:40507 (local address 127.1.32.67:47195)
0620 23:12:05.481484 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.481497 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.481538 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.481721 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.481722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.481723 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:05.481807 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.481809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.482697 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.482698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.483962 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.483964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.484715 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.484716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.484797 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.484807 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.484875 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.484922 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":151}
I20260620 23:12:05.485019 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.481437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42227 (local address 127.1.32.67:36473)
0620 23:12:05.481581 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.481590 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.481647 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.481775 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.481776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.481777 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:05.481833 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.481834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.482697 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.482698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.483962 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.483964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.484724 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.484726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.484808 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.484812 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.484877 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.484925 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":11,"thread_start_us":98,"threads_started":1}
I20260620 23:12:05.485345 18397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264deeff148a471ba472a1908b75f443" candidate_uuid: "6874189d51ee4e878da117069bfb2f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026ce0c39c944d6a9ca6db536c39d529" is_pre_election: true
I20260620 23:12:05.485471 18397 raft_consensus.cc:2454] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6874189d51ee4e878da117069bfb2f68 in term 0.
I20260620 23:12:05.485489 18263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264deeff148a471ba472a1908b75f443" candidate_uuid: "6874189d51ee4e878da117069bfb2f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" is_pre_election: true
I20260620 23:12:05.485581 18263 raft_consensus.cc:2454] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6874189d51ee4e878da117069bfb2f68 in term 0.
I20260620 23:12:05.485703 18467 leader_election.cc:302] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026ce0c39c944d6a9ca6db536c39d529, 6874189d51ee4e878da117069bfb2f68; no voters:
I20260620 23:12:05.485808 18602 raft_consensus.cc:2781] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:12:05.485857 18602 raft_consensus.cc:482] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:12:05.485877 18602 raft_consensus.cc:3037] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:05.486516 18602 raft_consensus.cc:504] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.486644 18602 leader_election.cc:288] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [CANDIDATE]: Term 1 election: Requested vote from peers 026ce0c39c944d6a9ca6db536c39d529 (127.1.32.66:40507), 5c1d4c1c2db447a5b4fd5faa633927ca (127.1.32.65:42227)
I20260620 23:12:05.486770 18397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264deeff148a471ba472a1908b75f443" candidate_uuid: "6874189d51ee4e878da117069bfb2f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026ce0c39c944d6a9ca6db536c39d529"
I20260620 23:12:05.486825 18397 raft_consensus.cc:3037] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:05.487033 18263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264deeff148a471ba472a1908b75f443" candidate_uuid: "6874189d51ee4e878da117069bfb2f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca"
I20260620 23:12:05.487100 18263 raft_consensus.cc:3037] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:05.487703 18397 raft_consensus.cc:2454] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6874189d51ee4e878da117069bfb2f68 in term 1.
I20260620 23:12:05.487936 18467 leader_election.cc:302] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026ce0c39c944d6a9ca6db536c39d529, 6874189d51ee4e878da117069bfb2f68; no voters:
I20260620 23:12:05.487958 18263 raft_consensus.cc:2454] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6874189d51ee4e878da117069bfb2f68 in term 1.
I20260620 23:12:05.488042 18602 raft_consensus.cc:2781] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:05.488217 18602 raft_consensus.cc:686] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 1 LEADER]: Becoming Leader. State: Replica: 6874189d51ee4e878da117069bfb2f68, State: Running, Role: LEADER
I20260620 23:12:05.488312 18602 consensus_queue.cc:237] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } }
I20260620 23:12:05.489017 18118 catalog_manager.cc:5495] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6874189d51ee4e878da117069bfb2f68 (127.1.32.67). New cstate: current_term: 1 leader_uuid: "6874189d51ee4e878da117069bfb2f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6874189d51ee4e878da117069bfb2f68" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 } health_report { overall_health: UNKNOWN } } }
I20260620 23:12:05.500133 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.498213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.1.32.67:41691)
0620 23:12:05.498267 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.498269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.498275 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:12:05.498298 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.498299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.498313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.498371 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:05.498472 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.498930 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.499602 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.499765 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.499809 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.499916 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:05.499994 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:05.500045 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.500073 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.500089 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:12:05.501667 18263 raft_consensus.cc:1261] T 264deeff148a471ba472a1908b75f443 P 5c1d4c1c2db447a5b4fd5faa633927ca [term 1 FOLLOWER]: Refusing update from remote peer 6874189d51ee4e878da117069bfb2f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:05.501688 18397 raft_consensus.cc:1261] T 264deeff148a471ba472a1908b75f443 P 026ce0c39c944d6a9ca6db536c39d529 [term 1 FOLLOWER]: Refusing update from remote peer 6874189d51ee4e878da117069bfb2f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:05.502000 18604 consensus_queue.cc:1035] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c1d4c1c2db447a5b4fd5faa633927ca" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:05.502127 18602 consensus_queue.cc:1035] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "026ce0c39c944d6a9ca6db536c39d529" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:05.504812 18609 mvcc.cc:205] Tried to move back new op lower bound from 7299060226052501504 to 7299060226000797696. Current Snapshot: MvccSnapshot[applied={T|T < 7299060226052501504}]
I20260620 23:12:05.505155 18607 mvcc.cc:205] Tried to move back new op lower bound from 7299060226052501504 to 7299060226000797696. Current Snapshot: MvccSnapshot[applied={T|T < 7299060226052501504}]
I20260620 23:12:05.505263 18608 mvcc.cc:205] Tried to move back new op lower bound from 7299060226052501504 to 7299060226000797696. Current Snapshot: MvccSnapshot[applied={T|T < 7299060226052501504 or (T in {7299060226052501504})}]
I20260620 23:12:05.507057 18118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:05.507141 18118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:05.507929 18118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8214e7e5e9ed4562bc9e32c33b39a87e: Sending DeleteTablet for 3 replicas of tablet 264deeff148a471ba472a1908b75f443
I20260620 23:12:05.508255 18377 tablet_service.cc:1514] Processing DeleteTablet for tablet 264deeff148a471ba472a1908b75f443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:05 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:50316
I20260620 23:12:05.508301 18243 tablet_service.cc:1514] Processing DeleteTablet for tablet 264deeff148a471ba472a1908b75f443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:05 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:34000
I20260620 23:12:05.508301 18511 tablet_service.cc:1514] Processing DeleteTablet for tablet 264deeff148a471ba472a1908b75f443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:05 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:42944
I20260620 23:12:05.508551 18619 tablet_replica.cc:330] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: stopping tablet replica
I20260620 23:12:05.508692 18619 raft_consensus.cc:2229] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:12:05.508796 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18180
I20260620 23:12:05.510474 18619 raft_consensus.cc:2258] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:05.514248 18619 ts_tablet_manager.cc:1905] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 23:12:05.514604 18102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:42227 (error 108)
I20260620 23:12:05.514806 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18314
I20260620 23:12:05.515875 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.515714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42227 (local address 127.0.0.1:34016)
0620 23:12:05.515777 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.515834 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:42227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260620 23:12:05.515987 18102 catalog_manager.cc:4584] TS 5c1d4c1c2db447a5b4fd5faa633927ca (127.1.32.65:42227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 264deeff148a471ba472a1908b75f443: Network error: Client connection negotiation failed: client connection to 127.1.32.65:42227: connect: Connection refused (error 111)
I20260620 23:12:05.516110 18619 ts_tablet_manager.cc:1918] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:05.516187 18619 log.cc:1196] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/264deeff148a471ba472a1908b75f443
I20260620 23:12:05.516462 18619 ts_tablet_manager.cc:1939] T 264deeff148a471ba472a1908b75f443 P 6874189d51ee4e878da117069bfb2f68: Deleting consensus metadata
I20260620 23:12:05.516892 18104 catalog_manager.cc:4841] TS 6874189d51ee4e878da117069bfb2f68 (127.1.32.67:41691): tablet 264deeff148a471ba472a1908b75f443 (table test-table [id=493eccbc20c8499fb32c78a88ee1adc7]) successfully deleted
I20260620 23:12:05.519987 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.519873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:40507 (local address 127.0.0.1:50332)
0620 23:12:05.519935 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.519957 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 23:12:05.519985 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18448
I20260620 23:12:05.524178 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18086
2026-06-20T23:12:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1887 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T23:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:05Z Disabled control of system clock
I20260620 23:12:05.550751 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40073
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:40073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:05.627727 18624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:05.627962 18624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:05.627982 18624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:05.629082 18624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:05.629119 18624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:05.629132 18624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:05.629165 18624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:05.629194 18624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:05.630425 18624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:40073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:05.630636 18624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:05.630926 18624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:05.635202 18629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.635305 18630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.635267 18624 server_base.cc:1034] running on GCE node
W20260620 23:12:05.635200 18632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.635749 18624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:05.636034 18624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:05.637190 18624 hybrid_clock.cc:648] HybridClock initialized: now 1781997125637176 us; error 25 us; skew 500 ppm
I20260620 23:12:05.638334 18624 webserver.cc:460] Webserver started at http://127.1.32.126:42237/ using document root <none> and password file <none>
I20260620 23:12:05.638545 18624 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:05.638605 18624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:05.638734 18624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:05.639674 18624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "ef92567659b64f7aad813bd26f437456"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "6887b84a73daaf99da78f57699398eca"
server_key_iv: "2d3f111282350d32e0820a072be9de7a"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.640012 18624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "ef92567659b64f7aad813bd26f437456"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "6887b84a73daaf99da78f57699398eca"
server_key_iv: "2d3f111282350d32e0820a072be9de7a"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.641177 18624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:05.641912 18638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.642071 18624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:05.642182 18624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "ef92567659b64f7aad813bd26f437456"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "6887b84a73daaf99da78f57699398eca"
server_key_iv: "2d3f111282350d32e0820a072be9de7a"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.642277 18624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:05.668680 18624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:05.668953 18624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:05.669093 18624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:05.672736 18624 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:40073
I20260620 23:12:05.672767 18690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:40073 every 8 connection(s)
I20260620 23:12:05.673074 18624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:05.673969 18691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:05.674631 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18624
I20260620 23:12:05.674732 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:05.674962 1153 external_mini_cluster.cc:1428] Setting key 42ad926059f085b3f052df5cb313a4e0
I20260620 23:12:05.676788 18691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456: Bootstrap starting.
I20260620 23:12:05.677337 18691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:05.677630 18691 log.cc:824] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:05.678381 18691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456: No bootstrap required, opened a new log
I20260620 23:12:05.680155 18691 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef92567659b64f7aad813bd26f437456" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40073 } }
I20260620 23:12:05.680275 18691 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:05.680296 18691 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef92567659b64f7aad813bd26f437456, State: Initialized, Role: FOLLOWER
I20260620 23:12:05.680310 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.675451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.1.32.126:40073)
0620 23:12:05.675627 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.675634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.675657 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:12:05.676048 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.676055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.676109 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.676213 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:05.676418 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.677269 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.678302 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.678724 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.678810 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.679501 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:05.679504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:05.679504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:05.679662 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:05.679692 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:05.679701 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:05.679749 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.679846 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.679941 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260620 23:12:05.680476 18691 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef92567659b64f7aad813bd26f437456" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40073 } }
I20260620 23:12:05.680553 18691 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:05.680593 18691 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:05.680635 18691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:05.681113 18691 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef92567659b64f7aad813bd26f437456" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40073 } }
I20260620 23:12:05.681209 18691 leader_election.cc:302] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef92567659b64f7aad813bd26f437456; no voters:
I20260620 23:12:05.681392 18691 leader_election.cc:288] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:05.681473 18696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:05.681620 18696 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [term 1 LEADER]: Becoming Leader. State: Replica: ef92567659b64f7aad813bd26f437456, State: Running, Role: LEADER
I20260620 23:12:05.681675 18691 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:05.681746 18696 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef92567659b64f7aad813bd26f437456" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40073 } }
I20260620 23:12:05.682070 18698 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef92567659b64f7aad813bd26f437456. Latest consensus state: current_term: 1 leader_uuid: "ef92567659b64f7aad813bd26f437456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef92567659b64f7aad813bd26f437456" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40073 } } }
I20260620 23:12:05.682192 18698 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:05.682070 18697 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef92567659b64f7aad813bd26f437456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef92567659b64f7aad813bd26f437456" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40073 } } }
I20260620 23:12:05.682369 18697 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:05.682703 18705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:05.683174 18705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:05.684607 18705 catalog_manager.cc:1347] Generated new cluster ID: 269f9278fe684f92a6edba397cec54e7
I20260620 23:12:05.684659 18705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:05.697314 18705 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:05.697777 18705 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:05.703469 18705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456: Generated new TSK 0
I20260620 23:12:05.703642 18705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:05.704769 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40073
--builtin_ntp_servers=127.1.32.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:05.783847 18715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:05.784053 18715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:05.784071 18715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:05.785159 18715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:05.785194 18715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:05.785218 18715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:12:05.786477 18715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:05.786698 18715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:05.787088 18715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:05.791893 18720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.792007 18715 server_base.cc:1034] running on GCE node
W20260620 23:12:05.791913 18721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.791920 18723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.792323 18715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:05.792568 18715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:05.793743 18715 hybrid_clock.cc:648] HybridClock initialized: now 1781997125793721 us; error 30 us; skew 500 ppm
I20260620 23:12:05.795162 18715 webserver.cc:460] Webserver started at http://127.1.32.65:41837/ using document root <none> and password file <none>
I20260620 23:12:05.795396 18715 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:05.795449 18715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:05.795565 18715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:05.796814 18715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "28c9b3c436864192860c9baeb3cd67d2"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "82e583436ce9158f1550458d953d8d9d"
server_key_iv: "db4c6b0f432ec05511053a3b09effc36"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.797183 18715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "28c9b3c436864192860c9baeb3cd67d2"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "82e583436ce9158f1550458d953d8d9d"
server_key_iv: "db4c6b0f432ec05511053a3b09effc36"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.798745 18715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:05.799607 18729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.799806 18715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:05.799896 18715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "28c9b3c436864192860c9baeb3cd67d2"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "82e583436ce9158f1550458d953d8d9d"
server_key_iv: "db4c6b0f432ec05511053a3b09effc36"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.799964 18715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:05.807793 18715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:05.807994 18715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:05.808079 18715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:05.808224 18715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:05.808456 18715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:05.808483 18715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.808501 18715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:05.808513 18715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.813817 18715 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:42967
I20260620 23:12:05.813901 18842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:42967 every 8 connection(s)
I20260620 23:12:05.814172 18715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:12:05.818624 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.814832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:41691 (local address 127.1.32.126:40073)
0620 23:12:05.814872 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.814874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.815136 (+ 262us) server_negotiation.cc:407] Connection header received
0620 23:12:05.815543 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.815544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.815556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.815605 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:05.816168 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.816632 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.817526 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.817702 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.817754 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.818410 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:05.818411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:05.818411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:05.818415 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:05.818428 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:05.818432 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:05.818476 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.818544 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.818567 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:05.818771 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.814842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40073 (local address 127.1.32.65:41691)
0620 23:12:05.815025 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.815054 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.815468 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.815652 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.815654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.815662 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:05.816106 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.816110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.816699 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.816700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.817453 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.817455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.817770 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.817771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.817866 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.818295 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:05.818295 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:05.818319 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:05.818338 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:05.818478 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:05.818481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:05.818482 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.818526 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.818594 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260620 23:12:05.819114 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18715
I20260620 23:12:05.819228 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:12:05.819386 18843 heartbeater.cc:347] Connected to a master server at 127.1.32.126:40073
I20260620 23:12:05.819429 1153 external_mini_cluster.cc:1428] Setting key a8cfa96946c33fa53f7a6fa7bf17a7b7
I20260620 23:12:05.819474 18843 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:05.819674 18843 heartbeater.cc:511] Master 127.1.32.126:40073 requested a full tablet report, sending...
I20260620 23:12:05.820163 18655 ts_manager.cc:194] Registered new tserver with Master: 28c9b3c436864192860c9baeb3cd67d2 (127.1.32.65:42967)
I20260620 23:12:05.820308 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40073
--builtin_ntp_servers=127.1.32.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:12:05.820818 18655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.65:41691
W20260620 23:12:05.896353 18846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:05.896535 18846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:05.896554 18846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:05.897596 18846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:05.897624 18846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:05.897646 18846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:12:05.898988 18846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:05.899158 18846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:05.899382 18846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:05.903990 18854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.903990 18851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:05.904011 18852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:05.904230 18846 server_base.cc:1034] running on GCE node
I20260620 23:12:05.904451 18846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:05.904675 18846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:05.905838 18846 hybrid_clock.cc:648] HybridClock initialized: now 1781997125905825 us; error 24 us; skew 500 ppm
I20260620 23:12:05.907166 18846 webserver.cc:460] Webserver started at http://127.1.32.66:40765/ using document root <none> and password file <none>
I20260620 23:12:05.907413 18846 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:05.907478 18846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:05.907612 18846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:05.908524 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "618de86095c848b08fdd4bea3571acbb"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "503812bf33f8c4c85961d506d45aa663"
server_key_iv: "9584c636ed4bf51cabccdab29b506382"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.908888 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "618de86095c848b08fdd4bea3571acbb"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "503812bf33f8c4c85961d506d45aa663"
server_key_iv: "9584c636ed4bf51cabccdab29b506382"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.910032 18846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:05.910658 18860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.910806 18846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:05.910923 18846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "618de86095c848b08fdd4bea3571acbb"
format_stamp: "Formatted at 2026-06-20 23:12:05 on dist-test-slave-fzjm"
server_key: "503812bf33f8c4c85961d506d45aa663"
server_key_iv: "9584c636ed4bf51cabccdab29b506382"
server_key_version: "encryptionkey@0"
I20260620 23:12:05.911005 18846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:05.928900 18846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:05.929169 18846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:05.929322 18846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:05.929541 18846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:05.929847 18846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:05.929910 18846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.929950 18846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:05.929977 18846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:05.935483 18846 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:43381
I20260620 23:12:05.935531 18973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:43381 every 8 connection(s)
I20260620 23:12:05.935804 18846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:12:05.940632 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.936475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:53231 (local address 127.1.32.126:40073)
0620 23:12:05.936521 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:05.936523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:05.936745 (+ 222us) server_negotiation.cc:407] Connection header received
0620 23:12:05.937229 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:05.937230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:05.937241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:05.937292 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:05.937926 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.938368 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.939412 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.939600 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.939685 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.940356 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:05.940357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:05.940357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:05.940361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:05.940394 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:05.940398 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:05.940452 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:05.940536 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 23:12:05.940560 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 23:12:05.940841 18976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:05.936441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40073 (local address 127.1.32.66:53231)
0620 23:12:05.936636 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:05.936665 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:05.937127 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:05.937354 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:05.937357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:05.937366 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:05.937823 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.937827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.938475 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.938477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.939323 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:05.939325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:05.939690 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:05.939691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:05.939811 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:05.940225 (+ 414us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:05.940226 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:05.940249 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:05.940268 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:05.940453 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:05.940458 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:05.940461 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:05.940537 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:12:05.940625 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260620 23:12:05.941325 18974 heartbeater.cc:347] Connected to a master server at 127.1.32.126:40073
I20260620 23:12:05.941394 18974 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:05.941570 18974 heartbeater.cc:511] Master 127.1.32.126:40073 requested a full tablet report, sending...
I20260620 23:12:05.941905 18655 ts_manager.cc:194] Registered new tserver with Master: 618de86095c848b08fdd4bea3571acbb (127.1.32.66:43381)
I20260620 23:12:05.942287 18655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.66:53231
I20260620 23:12:05.944756 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18846
I20260620 23:12:05.944834 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:12:05.944995 1153 external_mini_cluster.cc:1428] Setting key 7a12389519d2eee2734bff2cfe708c49
I20260620 23:12:05.945744 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40073
--builtin_ntp_servers=127.1.32.84:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:06.021533 18977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:06.021721 18977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:06.021741 18977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:06.022791 18977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:06.022822 18977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:06.022846 18977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:12:06.024209 18977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:06.024384 18977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:06.024602 18977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:06.029366 18983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:06.029379 18982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:06.029361 18985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:06.029611 18977 server_base.cc:1034] running on GCE node
I20260620 23:12:06.029826 18977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:06.030053 18977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:06.031190 18977 hybrid_clock.cc:648] HybridClock initialized: now 1781997126031163 us; error 50 us; skew 500 ppm
I20260620 23:12:06.032419 18977 webserver.cc:460] Webserver started at http://127.1.32.67:33703/ using document root <none> and password file <none>
I20260620 23:12:06.032672 18977 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:06.032738 18977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:06.032892 18977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:06.033921 18977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "7230f96ae5224c72946a28bbce806d78"
format_stamp: "Formatted at 2026-06-20 23:12:06 on dist-test-slave-fzjm"
server_key: "d057de7d873ece38d1b0df3125c1a515"
server_key_iv: "15d678d61e47597c05687a2b828f6fb2"
server_key_version: "encryptionkey@0"
I20260620 23:12:06.034258 18977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "7230f96ae5224c72946a28bbce806d78"
format_stamp: "Formatted at 2026-06-20 23:12:06 on dist-test-slave-fzjm"
server_key: "d057de7d873ece38d1b0df3125c1a515"
server_key_iv: "15d678d61e47597c05687a2b828f6fb2"
server_key_version: "encryptionkey@0"
I20260620 23:12:06.035604 18977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:06.036301 18991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:06.036491 18977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:06.036585 18977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "7230f96ae5224c72946a28bbce806d78"
format_stamp: "Formatted at 2026-06-20 23:12:06 on dist-test-slave-fzjm"
server_key: "d057de7d873ece38d1b0df3125c1a515"
server_key_iv: "15d678d61e47597c05687a2b828f6fb2"
server_key_version: "encryptionkey@0"
I20260620 23:12:06.036664 18977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:06.044628 18977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:06.044893 18977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:06.045028 18977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:06.045231 18977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:06.045527 18977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:06.045578 18977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:06.045614 18977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:06.045642 18977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:06.051079 18977 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:43209
I20260620 23:12:06.051117 19104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:43209 every 8 connection(s)
I20260620 23:12:06.051407 18977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:12:06.056734 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.052105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:42695 (local address 127.1.32.126:40073)
0620 23:12:06.052175 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.052178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.052396 (+ 218us) server_negotiation.cc:407] Connection header received
0620 23:12:06.052841 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.052842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.052853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.052904 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:06.053531 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.053985 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.055056 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.055250 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.055306 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.056403 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:06.056405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:06.056406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:06.056411 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:06.056427 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:06.056432 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:06.056490 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.056607 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.056650 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 23:12:06.056936 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.052073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40073 (local address 127.1.32.67:42695)
0620 23:12:06.052275 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:06.052314 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:06.052759 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:06.052955 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:06.052958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:06.052965 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:06.053443 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.053447 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.054064 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.054065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.054864 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.054866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.055333 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.055334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.055463 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.056216 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:06.056217 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:06.056250 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:06.056275 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:06.056513 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:06.056518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:06.056521 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:06.056580 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 23:12:06.056716 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260620 23:12:06.057401 19105 heartbeater.cc:347] Connected to a master server at 127.1.32.126:40073
I20260620 23:12:06.057489 19105 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:06.057627 19105 heartbeater.cc:511] Master 127.1.32.126:40073 requested a full tablet report, sending...
I20260620 23:12:06.057982 18655 ts_manager.cc:194] Registered new tserver with Master: 7230f96ae5224c72946a28bbce806d78 (127.1.32.67:43209)
I20260620 23:12:06.058347 18655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.67:42695
I20260620 23:12:06.059953 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 18977
I20260620 23:12:06.060031 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:12:06.060216 1153 external_mini_cluster.cc:1428] Setting key fa7df457ad14e412fb9af51b0feb8f3f
I20260620 23:12:06.060954 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:12:06.064831 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.062275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56280 (local address 127.1.32.126:40073)
0620 23:12:06.062340 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.062341 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.062413 (+ 72us) server_negotiation.cc:407] Connection header received
0620 23:12:06.062469 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.062469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.062478 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.062519 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:06.062665 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.063188 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.063861 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.064042 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.064096 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.064584 (+ 488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:06.064585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:06.064585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:06.064589 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:06.064601 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:06.064604 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:06.064652 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.064761 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.064779 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:12:06.066066 18655 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:12:06.067698 18655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:12:06.074061 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.069709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:43209 (local address 127.0.0.1:50614)
0620 23:12:06.069990 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:06.070030 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:06.070096 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:06.070519 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:06.070520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:06.070522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:06.070574 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.070576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.071814 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.071815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.072857 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.072859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.073758 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.073758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.073850 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.073852 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:12:06.073914 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 23:12:06.073977 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":59,"threads_started":1}
I20260620 23:12:06.074085 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.070113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36100 (local address 127.1.32.66:43381)
0620 23:12:06.070273 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.070277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.070289 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:12:06.070336 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.070338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.070353 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.070444 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:06.070658 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.071430 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.072856 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.073574 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.073642 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.073719 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.073955 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.073997 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260620 23:12:06.074368 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.069779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.1.32.67:43209)
0620 23:12:06.070330 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.070333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.070356 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:12:06.070386 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.070388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.070400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.070464 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:06.070646 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.071728 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.072932 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.073574 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.073642 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.073719 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.074260 (+ 541us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.074299 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":102,"threads_started":1}
I20260620 23:12:06.074061 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.069906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:43381 (local address 127.0.0.1:36100)
0620 23:12:06.070192 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:06.070200 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:06.070235 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:06.070523 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:06.070524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:06.070526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:06.070568 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.070570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.071524 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.071525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.072754 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.072756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.073756 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.073757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.073838 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.073850 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 23:12:06.073919 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:12:06.073977 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":28,"thread_start_us":71,"threads_started":1}
I20260620 23:12:06.074652 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.069711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42967 (local address 127.0.0.1:35392)
0620 23:12:06.070031 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:06.070040 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:06.070096 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:06.070330 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:06.070333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:06.070335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:06.070399 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.070401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.072949 (+ 2548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.072949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.073568 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.073569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.074410 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.074411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.074473 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.074477 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:06.074527 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 23:12:06.074566 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260620 23:12:06.074671 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.069803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.1.32.65:42967)
0620 23:12:06.069968 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.069972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.070172 (+ 200us) server_negotiation.cc:407] Connection header received
0620 23:12:06.070220 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.070223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.070239 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.070316 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:06.070530 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.071737 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.073660 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.074303 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.074403 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.074488 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.074549 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.074596 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260620 23:12:06.075217 19039 tablet_service.cc:1467] Processing CreateTablet for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 (DEFAULT_TABLE table=test-table [id=6ba8208d65aa40f2920c832e5b1e37af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:06.075349 18777 tablet_service.cc:1467] Processing CreateTablet for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 (DEFAULT_TABLE table=test-table [id=6ba8208d65aa40f2920c832e5b1e37af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:06.075579 19039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:06.075644 18777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:06.075592 18908 tablet_service.cc:1467] Processing CreateTablet for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 (DEFAULT_TABLE table=test-table [id=6ba8208d65aa40f2920c832e5b1e37af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:06.075909 18908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:06.078521 19119 tablet_bootstrap.cc:492] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Bootstrap starting.
I20260620 23:12:06.079015 19120 tablet_bootstrap.cc:492] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Bootstrap starting.
I20260620 23:12:06.079309 19119 tablet_bootstrap.cc:654] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:06.079620 19120 tablet_bootstrap.cc:654] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:06.079619 19119 log.cc:824] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:06.079917 19120 log.cc:824] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:06.080168 19121 tablet_bootstrap.cc:492] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2: Bootstrap starting.
I20260620 23:12:06.080366 19119 tablet_bootstrap.cc:492] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: No bootstrap required, opened a new log
I20260620 23:12:06.080442 19119 ts_tablet_manager.cc:1397] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:06.080595 19120 tablet_bootstrap.cc:492] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: No bootstrap required, opened a new log
I20260620 23:12:06.080675 19120 ts_tablet_manager.cc:1397] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 23:12:06.080771 19121 tablet_bootstrap.cc:654] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:06.081066 19121 log.cc:824] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:06.081818 19121 tablet_bootstrap.cc:492] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2: No bootstrap required, opened a new log
I20260620 23:12:06.081898 19121 ts_tablet_manager.cc:1397] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 23:12:06.082881 19120 raft_consensus.cc:348] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.083101 19120 raft_consensus.cc:374] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:06.083146 19120 raft_consensus.cc:729] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7230f96ae5224c72946a28bbce806d78, State: Initialized, Role: FOLLOWER
I20260620 23:12:06.083281 19120 consensus_queue.cc:260] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.083417 19119 raft_consensus.cc:348] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.083540 19120 ts_tablet_manager.cc:1428] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 23:12:06.083576 19119 raft_consensus.cc:374] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:06.083609 19119 raft_consensus.cc:729] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618de86095c848b08fdd4bea3571acbb, State: Initialized, Role: FOLLOWER
I20260620 23:12:06.083662 19105 heartbeater.cc:503] Master 127.1.32.126:40073 was elected leader, sending a full tablet report...
I20260620 23:12:06.083715 19119 consensus_queue.cc:260] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.083959 19119 ts_tablet_manager.cc:1428] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 23:12:06.084095 18974 heartbeater.cc:503] Master 127.1.32.126:40073 was elected leader, sending a full tablet report...
I20260620 23:12:06.084314 19121 raft_consensus.cc:348] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.084481 19121 raft_consensus.cc:374] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:06.084523 19121 raft_consensus.cc:729] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c9b3c436864192860c9baeb3cd67d2, State: Initialized, Role: FOLLOWER
I20260620 23:12:06.084625 19121 consensus_queue.cc:260] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.084847 19121 ts_tablet_manager.cc:1428] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:12:06.084875 18843 heartbeater.cc:503] Master 127.1.32.126:40073 was elected leader, sending a full tablet report...
W20260620 23:12:06.186291 18975 tablet.cc:2369] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:12:06.301904 19106 tablet.cc:2369] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:12:06.315026 18844 tablet.cc:2369] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:12:06.406754 19127 raft_consensus.cc:482] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:06.406876 19127 raft_consensus.cc:504] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.407361 19127 leader_election.cc:288] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7230f96ae5224c72946a28bbce806d78 (127.1.32.67:43209), 618de86095c848b08fdd4bea3571acbb (127.1.32.66:43381)
I20260620 23:12:06.411029 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.407770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:52955 (local address 127.1.32.66:43381)
0620 23:12:06.407811 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.407813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.407845 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:12:06.407914 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.407914 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.407921 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.407967 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:06.408078 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.408748 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.410153 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.410636 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.410708 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.410718 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.410889 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.410967 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:06.411031 19129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.407715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:43381 (local address 127.1.32.65:52955)
0620 23:12:06.407797 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:06.407806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:06.407854 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:06.407967 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:06.407967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:06.407968 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:06.408015 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.408016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.408821 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.408822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.410052 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.410054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.410727 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.410728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.410798 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.410802 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:06.410845 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 23:12:06.410938 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":48,"thread_start_us":35,"threads_started":1}
I20260620 23:12:06.411092 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.407586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:56471 (local address 127.1.32.67:43209)
0620 23:12:06.407661 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.407663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.407741 (+ 78us) server_negotiation.cc:407] Connection header received
0620 23:12:06.407841 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.407842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.407853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.407923 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:06.408078 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.408747 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.410153 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.410659 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.410711 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.410718 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.410980 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.411024 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:12:06.411129 19128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.407526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:43209 (local address 127.1.32.65:56471)
0620 23:12:06.407658 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:06.407669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:06.407757 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:06.407935 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:06.407935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:06.407937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:06.408013 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.408015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.408838 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.408839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.410051 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:06.410054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.410764 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.410766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:06.410853 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.410876 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 23:12:06.410971 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 23:12:06.411010 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":57,"thread_start_us":67,"threads_started":1}
I20260620 23:12:06.411437 18928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b42dcafc08d4cbc8d63fac4fb05e9e7" candidate_uuid: "28c9b3c436864192860c9baeb3cd67d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618de86095c848b08fdd4bea3571acbb" is_pre_election: true
I20260620 23:12:06.411497 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b42dcafc08d4cbc8d63fac4fb05e9e7" candidate_uuid: "28c9b3c436864192860c9baeb3cd67d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7230f96ae5224c72946a28bbce806d78" is_pre_election: true
I20260620 23:12:06.411594 18928 raft_consensus.cc:2454] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c9b3c436864192860c9baeb3cd67d2 in term 0.
I20260620 23:12:06.411640 19059 raft_consensus.cc:2454] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c9b3c436864192860c9baeb3cd67d2 in term 0.
I20260620 23:12:06.411875 18733 leader_election.cc:302] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c9b3c436864192860c9baeb3cd67d2, 7230f96ae5224c72946a28bbce806d78; no voters:
I20260620 23:12:06.411990 19127 raft_consensus.cc:2781] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:12:06.412077 19127 raft_consensus.cc:482] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:12:06.412132 19127 raft_consensus.cc:3037] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:06.412879 19127 raft_consensus.cc:504] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.413126 19127 leader_election.cc:288] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [CANDIDATE]: Term 1 election: Requested vote from peers 7230f96ae5224c72946a28bbce806d78 (127.1.32.67:43209), 618de86095c848b08fdd4bea3571acbb (127.1.32.66:43381)
I20260620 23:12:06.413200 19059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b42dcafc08d4cbc8d63fac4fb05e9e7" candidate_uuid: "28c9b3c436864192860c9baeb3cd67d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7230f96ae5224c72946a28bbce806d78"
I20260620 23:12:06.413291 19059 raft_consensus.cc:3037] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:06.413288 18928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b42dcafc08d4cbc8d63fac4fb05e9e7" candidate_uuid: "28c9b3c436864192860c9baeb3cd67d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "618de86095c848b08fdd4bea3571acbb"
I20260620 23:12:06.413364 18928 raft_consensus.cc:3037] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:06.414189 18928 raft_consensus.cc:2454] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c9b3c436864192860c9baeb3cd67d2 in term 1.
I20260620 23:12:06.414188 19059 raft_consensus.cc:2454] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c9b3c436864192860c9baeb3cd67d2 in term 1.
I20260620 23:12:06.414336 18733 leader_election.cc:302] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c9b3c436864192860c9baeb3cd67d2, 7230f96ae5224c72946a28bbce806d78; no voters:
I20260620 23:12:06.414430 19127 raft_consensus.cc:2781] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:06.414609 19127 raft_consensus.cc:686] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [term 1 LEADER]: Becoming Leader. State: Replica: 28c9b3c436864192860c9baeb3cd67d2, State: Running, Role: LEADER
I20260620 23:12:06.414937 19127 consensus_queue.cc:237] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } }
I20260620 23:12:06.415844 18654 catalog_manager.cc:5495] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28c9b3c436864192860c9baeb3cd67d2 (127.1.32.65). New cstate: current_term: 1 leader_uuid: "28c9b3c436864192860c9baeb3cd67d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c9b3c436864192860c9baeb3cd67d2" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 } health_report { overall_health: UNKNOWN } } }
I20260620 23:12:06.424721 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.422750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.1.32.65:42967)
0620 23:12:06.422801 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:06.422802 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:06.422810 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:06.422847 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:06.422848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:06.422861 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:06.422964 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:06.423071 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.423488 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.424185 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:06.424353 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:06.424400 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:06.424522 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:06.424604 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:06.424657 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:06.424668 (+ 11us) server_negotiation.cc:299] Negotiation successful
0620 23:12:06.424681 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 23:12:06.426261 19059 raft_consensus.cc:1261] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 1 FOLLOWER]: Refusing update from remote peer 28c9b3c436864192860c9baeb3cd67d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:06.426262 18928 raft_consensus.cc:1261] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 1 FOLLOWER]: Refusing update from remote peer 28c9b3c436864192860c9baeb3cd67d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:06.426535 19127 consensus_queue.cc:1035] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "618de86095c848b08fdd4bea3571acbb" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 43381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:06.426648 19130 consensus_queue.cc:1035] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 28c9b3c436864192860c9baeb3cd67d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7230f96ae5224c72946a28bbce806d78" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 43209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:06.428606 19133 mvcc.cc:205] Tried to move back new op lower bound from 7299060229839572992 to 7299060229796241408. Current Snapshot: MvccSnapshot[applied={T|T < 7299060229839572992}]
I20260620 23:12:06.428997 19135 mvcc.cc:205] Tried to move back new op lower bound from 7299060229839572992 to 7299060229796241408. Current Snapshot: MvccSnapshot[applied={T|T < 7299060229839572992}]
I20260620 23:12:06.429345 19134 mvcc.cc:205] Tried to move back new op lower bound from 7299060229839572992 to 7299060229796241408. Current Snapshot: MvccSnapshot[applied={T|T < 7299060229839572992}]
I20260620 23:12:06.430974 18655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:06.431054 18655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:06.431777 18655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef92567659b64f7aad813bd26f437456: Sending DeleteTablet for 3 replicas of tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7
I20260620 23:12:06.432083 18777 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:06 UTC) from {username='slave'} at 127.0.0.1:35392
I20260620 23:12:06.432098 19039 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:06 UTC) from {username='slave'} at 127.0.0.1:50614
I20260620 23:12:06.432112 18908 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:06 UTC) from {username='slave'} at 127.0.0.1:36100
I20260620 23:12:06.432693 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18715
I20260620 23:12:06.433296 19145 tablet_replica.cc:330] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: stopping tablet replica
I20260620 23:12:06.433399 19145 raft_consensus.cc:2229] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:06.433502 19145 raft_consensus.cc:2258] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:06.433683 19144 tablet_replica.cc:330] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: stopping tablet replica
I20260620 23:12:06.433774 19144 raft_consensus.cc:2229] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:06.433878 19144 raft_consensus.cc:2258] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:06.434291 19145 ts_tablet_manager.cc:1905] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:06.434307 19144 ts_tablet_manager.cc:1905] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:06.435869 19144 ts_tablet_manager.cc:1918] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:06.435925 19144 log.cc:1196] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/5b42dcafc08d4cbc8d63fac4fb05e9e7
I20260620 23:12:06.436021 19145 ts_tablet_manager.cc:1918] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:06.436093 19145 log.cc:1196] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/5b42dcafc08d4cbc8d63fac4fb05e9e7
I20260620 23:12:06.436257 19144 ts_tablet_manager.cc:1939] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 7230f96ae5224c72946a28bbce806d78: Deleting consensus metadata
I20260620 23:12:06.436403 19145 ts_tablet_manager.cc:1939] T 5b42dcafc08d4cbc8d63fac4fb05e9e7 P 618de86095c848b08fdd4bea3571acbb: Deleting consensus metadata
I20260620 23:12:06.436645 18642 catalog_manager.cc:4841] TS 7230f96ae5224c72946a28bbce806d78 (127.1.32.67:43209): tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 (table test-table [id=6ba8208d65aa40f2920c832e5b1e37af]) successfully deleted
I20260620 23:12:06.436770 18642 catalog_manager.cc:4841] TS 618de86095c848b08fdd4bea3571acbb (127.1.32.66:43381): tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7 (table test-table [id=6ba8208d65aa40f2920c832e5b1e37af]) successfully deleted
W20260620 23:12:06.439970 18641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:42967 (error 108)
I20260620 23:12:06.440155 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18846
I20260620 23:12:06.440902 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:06.440710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42967 (local address 127.0.0.1:35410)
0620 23:12:06.440750 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:06.440848 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260620 23:12:06.441031 18641 catalog_manager.cc:4584] TS 28c9b3c436864192860c9baeb3cd67d2 (127.1.32.65:42967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b42dcafc08d4cbc8d63fac4fb05e9e7: Network error: Client connection negotiation failed: client connection to 127.1.32.65:42967: connect: Connection refused (error 111)
I20260620 23:12:06.445820 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18977
I20260620 23:12:06.450677 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 18624
2026-06-20T23:12:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (925 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:06 dist-test-slave-fzjm krb5kdc[19149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:06 dist-test-slave-fzjm krb5kdc[19149](info): set up 2 sockets
Jun 20 23:12:06 dist-test-slave-fzjm krb5kdc[19149](info): commencing operation
krb5kdc: starting...
W20260620 23:12:07.474555 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:07 dist-test-slave-fzjm krb5kdc[19149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:07.585688 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:42815
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:42815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:07.662660 19162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:07.662880 19162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:07.662925 19162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:07.664005 19162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:07.664072 19162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:07.664094 19162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:07.664115 19162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:07.664141 19162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:07.665520 19162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:42815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:42815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:07.665776 19162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:07.665990 19162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:07.670831 19170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:07.670843 19167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:07.670909 19162 server_base.cc:1034] running on GCE node
W20260620 23:12:07.670836 19168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:07.671478 19162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:07.671824 19162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:07.672974 19162 hybrid_clock.cc:648] HybridClock initialized: now 1781997127672964 us; error 57 us; skew 500 ppm
Jun 20 23:12:07 dist-test-slave-fzjm krb5kdc[19149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:07.675498 19162 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:07.675937 19162 webserver.cc:460] Webserver started at http://127.1.32.126:33161/ using document root <none> and password file <none>
I20260620 23:12:07.676164 19162 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:07.676231 19162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:07.676379 19162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:07.677408 19162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "1e3d7072ae554089934ebb884344c8fa"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "723a1e62ab94038df98c2ce1770d010c"
server_key_iv: "eb03e19c4214b9538547e878a3d7942a"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.677762 19162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "1e3d7072ae554089934ebb884344c8fa"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "723a1e62ab94038df98c2ce1770d010c"
server_key_iv: "eb03e19c4214b9538547e878a3d7942a"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.679090 19162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:07.679787 19177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:07.679961 19162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:07.680076 19162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "1e3d7072ae554089934ebb884344c8fa"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "723a1e62ab94038df98c2ce1770d010c"
server_key_iv: "eb03e19c4214b9538547e878a3d7942a"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.680150 19162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:07.694465 19162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:07.696717 19162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:07.696867 19162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:07.700651 19162 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:42815
I20260620 23:12:07.700690 19229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:42815 every 8 connection(s)
I20260620 23:12:07.701051 19162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:07.701870 19230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:07.704125 19230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa: Bootstrap starting.
I20260620 23:12:07.704694 19230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:07.704989 19230 log.cc:824] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:07.705639 19230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa: No bootstrap required, opened a new log
I20260620 23:12:07.707186 19230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3d7072ae554089934ebb884344c8fa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42815 } }
I20260620 23:12:07.707316 19230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:07.707338 19230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3d7072ae554089934ebb884344c8fa, State: Initialized, Role: FOLLOWER
I20260620 23:12:07.707501 19230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3d7072ae554089934ebb884344c8fa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42815 } }
I20260620 23:12:07.707580 19230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:07.707631 19230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:07.707679 19230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:07.708166 19230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3d7072ae554089934ebb884344c8fa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42815 } }
I20260620 23:12:07.708272 19230 leader_election.cc:302] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3d7072ae554089934ebb884344c8fa; no voters:
I20260620 23:12:07.708462 19230 leader_election.cc:288] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:07.708488 19233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:07.708674 19233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [term 1 LEADER]: Becoming Leader. State: Replica: 1e3d7072ae554089934ebb884344c8fa, State: Running, Role: LEADER
I20260620 23:12:07.708730 19230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:07.708760 19233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3d7072ae554089934ebb884344c8fa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42815 } }
I20260620 23:12:07.709187 19235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e3d7072ae554089934ebb884344c8fa. Latest consensus state: current_term: 1 leader_uuid: "1e3d7072ae554089934ebb884344c8fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3d7072ae554089934ebb884344c8fa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42815 } } }
I20260620 23:12:07.709273 19235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:07.709326 19234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e3d7072ae554089934ebb884344c8fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3d7072ae554089934ebb884344c8fa" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42815 } } }
I20260620 23:12:07.709415 19234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:07.709689 19242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:07.709905 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 19162
I20260620 23:12:07.710021 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:07.710222 1153 external_mini_cluster.cc:1428] Setting key 5810344881be29a7d3a606cb5d272b26
I20260620 23:12:07.710220 19242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:07.712077 19242 catalog_manager.cc:1347] Generated new cluster ID: 07f55aca1e7746bab83c9f76710b648e
I20260620 23:12:07.712131 19242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:12:07 dist-test-slave-fzjm krb5kdc[19149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:07.720657 19242 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:07.721292 19242 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:07.721987 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:07.710786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.1.32.126:42815)
0620 23:12:07.711080 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:07.711090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:07.711124 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:12:07.711231 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:07.711237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:07.711297 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:07.711455 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:07.712257 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:07.713636 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:07.714714 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:07.715278 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:07.715371 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:07.718077 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:07.718081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:07.718082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:07.718093 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:07.720224 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:07.720646 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:07.720650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:07.720650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:07.720781 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:07.720988 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:07.720989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:07.720989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:07.721287 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:07.721446 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:07.721626 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 23:12:07.721764 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":235,"thread_start_us":79,"threads_started":1}
I20260620 23:12:07.725867 19242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa: Generated new TSK 0
I20260620 23:12:07.726034 19242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:12:07.751220 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42815
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:07.829818 19256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:07.829996 19256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:07.830017 19256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:07.831096 19256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:07.831132 19256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:07.831156 19256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:12:07.832566 19256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:07.832809 19256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:07.833017 19256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:07.837323 19261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:07.837323 19264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:07.837409 19262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:07.837534 19256 server_base.cc:1034] running on GCE node
I20260620 23:12:07.837760 19256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:07.837997 19256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:07.839143 19256 hybrid_clock.cc:648] HybridClock initialized: now 1781997127839135 us; error 36 us; skew 500 ppm
Jun 20 23:12:07 dist-test-slave-fzjm krb5kdc[19149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:07.841341 19256 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:12:07.841753 19256 webserver.cc:460] Webserver started at http://127.1.32.65:44133/ using document root <none> and password file <none>
I20260620 23:12:07.841977 19256 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:07.842039 19256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:07.842173 19256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:07.843156 19256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "345f743a927548f1871a80da099d4af9"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "640a7c38a8fa97ab00ab9d3047b6b5ee"
server_key_iv: "70ce60d03613dfe37fe10f6bf0dff138"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.843503 19256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "345f743a927548f1871a80da099d4af9"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "640a7c38a8fa97ab00ab9d3047b6b5ee"
server_key_iv: "70ce60d03613dfe37fe10f6bf0dff138"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.844763 19256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:07.845434 19271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:07.845600 19256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:07.845685 19256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "345f743a927548f1871a80da099d4af9"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "640a7c38a8fa97ab00ab9d3047b6b5ee"
server_key_iv: "70ce60d03613dfe37fe10f6bf0dff138"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.845754 19256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:07.864668 19256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:07.866845 19256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:07.867028 19256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:07.867254 19256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:07.867511 19256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:07.867538 19256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:07.867556 19256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:07.867568 19256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:07.872713 19256 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:33393
I20260620 23:12:07.872779 19384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:33393 every 8 connection(s)
I20260620 23:12:07.873030 19256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:12:07.875675 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 19256
I20260620 23:12:07.875785 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:12:07.875968 1153 external_mini_cluster.cc:1428] Setting key 4e20561282d0bd812a81b71a6d9c9fc4
Jun 20 23:12:07 dist-test-slave-fzjm krb5kdc[19149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:07.882052 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:07.873659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:42613 (local address 127.1.32.126:42815)
0620 23:12:07.873718 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:07.873720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:07.873901 (+ 181us) server_negotiation.cc:407] Connection header received
0620 23:12:07.874300 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:07.874301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:07.874311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:07.874358 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:07.875174 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:07.875648 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:07.876917 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:07.877128 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:07.877230 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:07.879231 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:07.879233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:07.879234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:07.879241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:07.880648 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:07.881073 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:07.881074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:07.881074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:07.881141 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:07.881397 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:07.881398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:07.881399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:07.881515 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:07.881585 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:07.881857 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 23:12:07.881961 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:12:07.882185 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:07.873607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42815 (local address 127.1.32.65:42613)
0620 23:12:07.873769 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:07.873806 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:07.874229 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:07.874419 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:07.874422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:07.874938 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:07.875105 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:07.875109 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:07.875744 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:07.875745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:07.876816 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:07.876818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:07.877240 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:07.877241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:07.877400 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:07.877879 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:07.877888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:07.879139 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:07.880753 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:07.880759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:07.880761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:07.880987 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:07.881207 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:07.881208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:07.881209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:07.881316 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:07.881588 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:07.881595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:07.881726 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 23:12:07.881824 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 23:12:07.881972 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260620 23:12:07.882623 19385 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42815
I20260620 23:12:07.882735 19385 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:07.882931 19385 heartbeater.cc:511] Master 127.1.32.126:42815 requested a full tablet report, sending...
I20260620 23:12:07.883462 19194 ts_manager.cc:194] Registered new tserver with Master: 345f743a927548f1871a80da099d4af9 (127.1.32.65:33393)
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:12:07.884112 19194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:42613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:12:07.898727 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42815
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:07.974476 19390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:07.974643 19390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:07.974663 19390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:07.975772 19390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:07.975811 19390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:07.975843 19390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:12:07.977108 19390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:07.977288 19390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:07.977451 19390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:07.982160 19398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:07.982197 19395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:07.982239 19390 server_base.cc:1034] running on GCE node
W20260620 23:12:07.982197 19396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:07.982668 19390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:07.982939 19390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:07.984098 19390 hybrid_clock.cc:648] HybridClock initialized: now 1781997127984077 us; error 38 us; skew 500 ppm
Jun 20 23:12:07 dist-test-slave-fzjm krb5kdc[19149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:07.986392 19390 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:12:07.986881 19390 webserver.cc:460] Webserver started at http://127.1.32.66:35675/ using document root <none> and password file <none>
I20260620 23:12:07.987175 19390 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:07.987244 19390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:07.987383 19390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:07.988365 19390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "bef97a69cf4c483e898979af852d15ea"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "844d35ebb82ad08a122fd5424eeeb85c"
server_key_iv: "6f788383715fb88ade345815f2b2afac"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.988722 19390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "bef97a69cf4c483e898979af852d15ea"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "844d35ebb82ad08a122fd5424eeeb85c"
server_key_iv: "6f788383715fb88ade345815f2b2afac"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.989980 19390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:12:07.990996 19405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:07.991154 19390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:07.991243 19390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "bef97a69cf4c483e898979af852d15ea"
format_stamp: "Formatted at 2026-06-20 23:12:07 on dist-test-slave-fzjm"
server_key: "844d35ebb82ad08a122fd5424eeeb85c"
server_key_iv: "6f788383715fb88ade345815f2b2afac"
server_key_version: "encryptionkey@0"
I20260620 23:12:07.991328 19390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:08.007709 19390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:08.009402 19390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:08.009541 19390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:08.009776 19390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:08.010068 19390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:08.010118 19390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.010146 19390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:08.010171 19390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.015829 19390 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:45915
I20260620 23:12:08.015865 19518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:45915 every 8 connection(s)
I20260620 23:12:08.016134 19390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
Jun 20 23:12:08 dist-test-slave-fzjm krb5kdc[19149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:08.022842 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 19390
I20260620 23:12:08.022964 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:12:08.023161 1153 external_mini_cluster.cc:1428] Setting key ae671fc19200faa03805ff6864c49276
I20260620 23:12:08.024441 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.016753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:48485 (local address 127.1.32.126:42815)
0620 23:12:08.016879 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.016881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.016962 (+ 81us) server_negotiation.cc:407] Connection header received
0620 23:12:08.017329 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.017330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.017350 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.017394 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.018158 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.018646 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.019575 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.019772 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.019858 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.021661 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.021663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.021663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:08.021670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.023174 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:08.023569 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:08.023571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:08.023571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:08.023631 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:08.023871 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:08.023871 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:08.023872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:08.023974 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.024034 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.024262 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.024357 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260620 23:12:08.024600 19521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.016706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42815 (local address 127.1.32.66:48485)
0620 23:12:08.016845 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.016884 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.017254 (+ 370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.017437 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.017439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.017926 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:08.018081 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.018085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.018723 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.018724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.019492 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.019494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.019880 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.019882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.020004 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.020349 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:08.020355 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:08.021566 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:08.023267 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:08.023272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:08.023274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:08.023478 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:08.023692 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:08.023692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:08.023693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:08.023802 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:08.024034 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:08.024040 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:08.024155 (+ 115us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.024243 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.024404 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260620 23:12:08.024999 19519 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42815
I20260620 23:12:08.025100 19519 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:08.025272 19519 heartbeater.cc:511] Master 127.1.32.126:42815 requested a full tablet report, sending...
I20260620 23:12:08.025576 19194 ts_manager.cc:194] Registered new tserver with Master: bef97a69cf4c483e898979af852d15ea (127.1.32.66:45915)
I20260620 23:12:08.025930 19194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:48485
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:12:08.045491 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42815
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:08.123484 19524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:08.123661 19524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:08.123685 19524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:08.124701 19524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:08.124737 19524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:08.124764 19524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:12:08.126096 19524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:08.126259 19524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:08.126435 19524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:08.131189 19530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.131201 19529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.131237 19532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:08.131392 19524 server_base.cc:1034] running on GCE node
I20260620 23:12:08.131592 19524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:08.131876 19524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:08.133031 19524 hybrid_clock.cc:648] HybridClock initialized: now 1781997128133016 us; error 45 us; skew 500 ppm
Jun 20 23:12:08 dist-test-slave-fzjm krb5kdc[19149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997128, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:08.135533 19524 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:12:08.135972 19524 webserver.cc:460] Webserver started at http://127.1.32.67:35435/ using document root <none> and password file <none>
I20260620 23:12:08.136224 19524 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:08.136286 19524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:08.136426 19524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:08.137399 19524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "0892b8d71b054a92afc215b1a14bcf15"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "93146b4f260a3cff1fd9bd5e927eaeb5"
server_key_iv: "438fb0d10c936e3f40662d7db9bb95dd"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.137713 19524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "0892b8d71b054a92afc215b1a14bcf15"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "93146b4f260a3cff1fd9bd5e927eaeb5"
server_key_iv: "438fb0d10c936e3f40662d7db9bb95dd"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.138947 19524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:12:08.139715 19539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.139887 19524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:08.139969 19524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "0892b8d71b054a92afc215b1a14bcf15"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "93146b4f260a3cff1fd9bd5e927eaeb5"
server_key_iv: "438fb0d10c936e3f40662d7db9bb95dd"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.140040 19524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:08.146821 19524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:08.149164 19524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:08.149327 19524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:08.149559 19524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:08.149889 19524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:08.149940 19524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.149976 19524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:08.150004 19524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.155683 19524 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:36099
I20260620 23:12:08.155745 19652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:36099 every 8 connection(s)
I20260620 23:12:08.156025 19524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:12:08.159715 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 19524
I20260620 23:12:08.159860 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:12:08.160054 1153 external_mini_cluster.cc:1428] Setting key b93e41650c2016d535f39774b854849f
Jun 20 23:12:08 dist-test-slave-fzjm krb5kdc[19149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997128, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:08.164763 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.156700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:35045 (local address 127.1.32.126:42815)
0620 23:12:08.156763 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.156766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.156991 (+ 225us) server_negotiation.cc:407] Connection header received
0620 23:12:08.157455 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.157456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.157465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.157537 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.158399 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.158874 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.159843 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.160129 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.160199 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.162286 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.162288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.162288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:08.162295 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.163468 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:08.163874 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:08.163875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:08.163876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:08.163934 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:08.164166 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:08.164167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:08.164167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:08.164276 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.164354 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.164604 (+ 250us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.164676 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:12:08.164947 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.156683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42815 (local address 127.1.32.67:35045)
0620 23:12:08.156842 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.156879 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.157373 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.157586 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.157589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.158126 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:08.158301 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.158305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.158998 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.158999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.159747 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.159749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.160206 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.160207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.160329 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.160805 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:08.160813 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:08.162195 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:08.163560 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:08.163567 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:08.163568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:08.163789 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:08.163998 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:08.163999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:08.163999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:08.164101 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:08.164345 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:08.164351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:08.164481 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.164583 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.164725 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
I20260620 23:12:08.165381 19653 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42815
I20260620 23:12:08.165452 19653 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:08.165625 19653 heartbeater.cc:511] Master 127.1.32.126:42815 requested a full tablet report, sending...
I20260620 23:12:08.165999 19194 ts_manager.cc:194] Registered new tserver with Master: 0892b8d71b054a92afc215b1a14bcf15 (127.1.32.67:36099)
I20260620 23:12:08.166304 19194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:35045
I20260620 23:12:08.171895 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:12:08.178550 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.173065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.1.32.126:42815)
0620 23:12:08.173122 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.173124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.173249 (+ 125us) server_negotiation.cc:407] Connection header received
0620 23:12:08.173270 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.173271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.173279 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.173330 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.173835 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.174259 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.174990 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.175158 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.175203 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.176302 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.176303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.176304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:08.176311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.177629 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:08.177947 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:08.177948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:08.177948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:08.177994 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:08.178141 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:08.178142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:08.178142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:08.178237 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.178305 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.178392 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.178435 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:12:08.180073 19194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:12:08.182030 19194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:12:08.187940 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.184122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.1.32.67:36099)
0620 23:12:08.184524 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.184527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.184533 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:12:08.184556 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.184558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.184567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.184644 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.184767 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.185517 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.186768 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.187565 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.187629 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.187674 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.187831 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.187866 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":75,"threads_started":1}
I20260620 23:12:08.187944 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.184063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:36099 (local address 127.0.0.1:52550)
0620 23:12:08.184226 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.184263 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.184342 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.184649 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.184651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.184653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.184716 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.184717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.185617 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.185618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.186671 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.186674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.187641 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.187641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.187711 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.187722 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.187791 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.187855 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":43,"threads_started":1}
I20260620 23:12:08.188144 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.184107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45915 (local address 127.0.0.1:35968)
0620 23:12:08.184341 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.184353 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.184410 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.184983 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.184984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.184987 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.185017 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.185018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.186275 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.186276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.187203 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.187205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.187950 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.187951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.188014 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.188017 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.188048 (+ 31us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.188085 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":59,"threads_started":1}
I20260620 23:12:08.188491 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.184231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:33393 (local address 127.0.0.1:36756)
0620 23:12:08.184370 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.184379 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.184416 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.184786 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.184787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.184789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.184829 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.184830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.186151 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.186152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.187259 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.187262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.188264 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.188265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.188329 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.188333 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.188382 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.188424 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":23,"thread_start_us":57,"threads_started":1}
I20260620 23:12:08.188477 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.184215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.1.32.66:45915)
0620 23:12:08.184629 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.184632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.184799 (+ 167us) server_negotiation.cc:407] Connection header received
0620 23:12:08.184847 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.184849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.184864 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.185083 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.185154 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.186171 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.187284 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.187855 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.188281 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.188347 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.188372 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.188408 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":73,"threads_started":1}
I20260620 23:12:08.188870 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.184431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.1.32.65:33393)
0620 23:12:08.184574 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.184578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.184589 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:12:08.184624 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.184627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.184641 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.184713 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.184905 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.186048 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.187352 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.188122 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.188182 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.188230 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.188767 (+ 537us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.188812 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260620 23:12:08.188907 19587 tablet_service.cc:1467] Processing CreateTablet for tablet aaee7c6830e64fefa378be0078fdfd8e (DEFAULT_TABLE table=test-table [id=ae045be09b2f4f3db461e63490471007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:08.189116 19453 tablet_service.cc:1467] Processing CreateTablet for tablet aaee7c6830e64fefa378be0078fdfd8e (DEFAULT_TABLE table=test-table [id=ae045be09b2f4f3db461e63490471007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:08.189232 19587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaee7c6830e64fefa378be0078fdfd8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:08.189364 19453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaee7c6830e64fefa378be0078fdfd8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:08.189399 19319 tablet_service.cc:1467] Processing CreateTablet for tablet aaee7c6830e64fefa378be0078fdfd8e (DEFAULT_TABLE table=test-table [id=ae045be09b2f4f3db461e63490471007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:08.189631 19319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaee7c6830e64fefa378be0078fdfd8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:08.192107 19667 tablet_bootstrap.cc:492] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9: Bootstrap starting.
I20260620 23:12:08.192595 19668 tablet_bootstrap.cc:492] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Bootstrap starting.
I20260620 23:12:08.192787 19667 tablet_bootstrap.cc:654] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:08.193312 19668 tablet_bootstrap.cc:654] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:08.193413 19667 log.cc:824] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:08.193938 19668 log.cc:824] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:08.194167 19667 tablet_bootstrap.cc:492] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9: No bootstrap required, opened a new log
I20260620 23:12:08.194239 19667 ts_tablet_manager.cc:1397] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:08.194247 19669 tablet_bootstrap.cc:492] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Bootstrap starting.
I20260620 23:12:08.194583 19668 tablet_bootstrap.cc:492] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: No bootstrap required, opened a new log
I20260620 23:12:08.194657 19668 ts_tablet_manager.cc:1397] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:08.194911 19669 tablet_bootstrap.cc:654] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:08.195464 19669 log.cc:824] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:08.196234 19669 tablet_bootstrap.cc:492] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: No bootstrap required, opened a new log
I20260620 23:12:08.196305 19669 ts_tablet_manager.cc:1397] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:08.196259 19668 raft_consensus.cc:348] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.196465 19668 raft_consensus.cc:374] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:08.196507 19668 raft_consensus.cc:729] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0892b8d71b054a92afc215b1a14bcf15, State: Initialized, Role: FOLLOWER
I20260620 23:12:08.196503 19667 raft_consensus.cc:348] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.196619 19668 consensus_queue.cc:260] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.196671 19667 raft_consensus.cc:374] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:08.196707 19667 raft_consensus.cc:729] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345f743a927548f1871a80da099d4af9, State: Initialized, Role: FOLLOWER
I20260620 23:12:08.196837 19667 consensus_queue.cc:260] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.196889 19668 ts_tablet_manager.cc:1428] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:08.197010 19653 heartbeater.cc:503] Master 127.1.32.126:42815 was elected leader, sending a full tablet report...
I20260620 23:12:08.197093 19667 ts_tablet_manager.cc:1428] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:12:08.197242 19385 heartbeater.cc:503] Master 127.1.32.126:42815 was elected leader, sending a full tablet report...
I20260620 23:12:08.198251 19669 raft_consensus.cc:348] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.198424 19669 raft_consensus.cc:374] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:08.198475 19669 raft_consensus.cc:729] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef97a69cf4c483e898979af852d15ea, State: Initialized, Role: FOLLOWER
I20260620 23:12:08.198576 19669 consensus_queue.cc:260] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.198763 19669 ts_tablet_manager.cc:1428] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:12:08.198830 19519 heartbeater.cc:503] Master 127.1.32.126:42815 was elected leader, sending a full tablet report...
I20260620 23:12:08.203840 19675 raft_consensus.cc:482] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:08.203927 19675 raft_consensus.cc:504] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.204282 19675 leader_election.cc:288] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0892b8d71b054a92afc215b1a14bcf15 (127.1.32.67:36099), 345f743a927548f1871a80da099d4af9 (127.1.32.65:33393)
I20260620 23:12:08.207315 19521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.204431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:36099 (local address 127.1.32.66:57611)
0620 23:12:08.204476 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.204490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.204541 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.204705 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.204706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.204708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.204774 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.204775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.205651 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.205652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.206379 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.206381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.207040 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.207041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.207122 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.207126 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.207204 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.207247 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":1}
I20260620 23:12:08.207317 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.204490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:57611 (local address 127.1.32.67:36099)
0620 23:12:08.204549 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.204552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.204560 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:08.204588 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.204589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.204600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.204658 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.204843 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.205555 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.206512 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.206941 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.207038 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.207047 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.207211 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.207246 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:12:08.207475 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.204685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:50935 (local address 127.1.32.65:33393)
0620 23:12:08.204729 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.204731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.204931 (+ 200us) server_negotiation.cc:407] Connection header received
0620 23:12:08.204954 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.204955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.204964 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.205026 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.205111 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.205757 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.206658 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.207106 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.207192 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.207201 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.207363 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.207423 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:12:08.207475 19676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.204556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:33393 (local address 127.1.32.66:50935)
0620 23:12:08.204873 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.204881 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.204920 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.205028 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.205029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.205030 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.205069 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.205070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.205838 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.205839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.206573 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.206575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.207197 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.207198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.207268 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.207274 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.207355 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.207393 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":82,"threads_started":1}
I20260620 23:12:08.208112 19339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee7c6830e64fefa378be0078fdfd8e" candidate_uuid: "bef97a69cf4c483e898979af852d15ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345f743a927548f1871a80da099d4af9" is_pre_election: true
I20260620 23:12:08.208123 19607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee7c6830e64fefa378be0078fdfd8e" candidate_uuid: "bef97a69cf4c483e898979af852d15ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0892b8d71b054a92afc215b1a14bcf15" is_pre_election: true
I20260620 23:12:08.208231 19339 raft_consensus.cc:2454] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef97a69cf4c483e898979af852d15ea in term 0.
I20260620 23:12:08.208231 19607 raft_consensus.cc:2454] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bef97a69cf4c483e898979af852d15ea in term 0.
I20260620 23:12:08.208446 19408 leader_election.cc:302] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345f743a927548f1871a80da099d4af9, bef97a69cf4c483e898979af852d15ea; no voters:
I20260620 23:12:08.208547 19675 raft_consensus.cc:2781] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:12:08.208596 19675 raft_consensus.cc:482] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:12:08.208617 19675 raft_consensus.cc:3037] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:08.209261 19675 raft_consensus.cc:504] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.209365 19675 leader_election.cc:288] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [CANDIDATE]: Term 1 election: Requested vote from peers 0892b8d71b054a92afc215b1a14bcf15 (127.1.32.67:36099), 345f743a927548f1871a80da099d4af9 (127.1.32.65:33393)
I20260620 23:12:08.209502 19607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee7c6830e64fefa378be0078fdfd8e" candidate_uuid: "bef97a69cf4c483e898979af852d15ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0892b8d71b054a92afc215b1a14bcf15"
I20260620 23:12:08.209529 19339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee7c6830e64fefa378be0078fdfd8e" candidate_uuid: "bef97a69cf4c483e898979af852d15ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345f743a927548f1871a80da099d4af9"
I20260620 23:12:08.209568 19607 raft_consensus.cc:3037] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:08.209590 19339 raft_consensus.cc:3037] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:08.210302 19607 raft_consensus.cc:2454] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef97a69cf4c483e898979af852d15ea in term 1.
I20260620 23:12:08.210325 19339 raft_consensus.cc:2454] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bef97a69cf4c483e898979af852d15ea in term 1.
I20260620 23:12:08.210489 19408 leader_election.cc:302] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0892b8d71b054a92afc215b1a14bcf15, bef97a69cf4c483e898979af852d15ea; no voters:
I20260620 23:12:08.210610 19675 raft_consensus.cc:2781] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:08.210788 19675 raft_consensus.cc:686] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 1 LEADER]: Becoming Leader. State: Replica: bef97a69cf4c483e898979af852d15ea, State: Running, Role: LEADER
I20260620 23:12:08.210883 19675 consensus_queue.cc:237] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } }
I20260620 23:12:08.211655 19193 catalog_manager.cc:5495] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea reported cstate change: term changed from 0 to 1, leader changed from <none> to bef97a69cf4c483e898979af852d15ea (127.1.32.66). New cstate: current_term: 1 leader_uuid: "bef97a69cf4c483e898979af852d15ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bef97a69cf4c483e898979af852d15ea" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45915 } health_report { overall_health: HEALTHY } } }
I20260620 23:12:08.217760 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.215705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.1.32.66:45915)
0620 23:12:08.215742 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.215744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.215751 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:08.215838 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.215839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.215853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.215910 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:08.216022 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.216435 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.217104 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.217257 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.217313 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.217443 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:08.217527 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:08.217607 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.217642 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.217649 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:08.219340 19339 raft_consensus.cc:1261] T aaee7c6830e64fefa378be0078fdfd8e P 345f743a927548f1871a80da099d4af9 [term 1 FOLLOWER]: Refusing update from remote peer bef97a69cf4c483e898979af852d15ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:08.219340 19607 raft_consensus.cc:1261] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 1 FOLLOWER]: Refusing update from remote peer bef97a69cf4c483e898979af852d15ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:08.219662 19677 consensus_queue.cc:1035] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "0892b8d71b054a92afc215b1a14bcf15" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 36099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:08.219810 19675 consensus_queue.cc:1035] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "345f743a927548f1871a80da099d4af9" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 33393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:08.222055 19680 mvcc.cc:205] Tried to move back new op lower bound from 7299060237183680512 to 7299060237152526336. Current Snapshot: MvccSnapshot[applied={T|T < 7299060237183680512}]
I20260620 23:12:08.222515 19682 mvcc.cc:205] Tried to move back new op lower bound from 7299060237183680512 to 7299060237152526336. Current Snapshot: MvccSnapshot[applied={T|T < 7299060237183680512}]
I20260620 23:12:08.222790 19681 mvcc.cc:205] Tried to move back new op lower bound from 7299060237183680512 to 7299060237152526336. Current Snapshot: MvccSnapshot[applied={T|T < 7299060237183680512}]
I20260620 23:12:08.224702 19193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:08.224792 19193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:08.225517 19193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e3d7072ae554089934ebb884344c8fa: Sending DeleteTablet for 3 replicas of tablet aaee7c6830e64fefa378be0078fdfd8e
I20260620 23:12:08.225874 19453 tablet_service.cc:1514] Processing DeleteTablet for tablet aaee7c6830e64fefa378be0078fdfd8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:08 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:35968
I20260620 23:12:08.225898 19319 tablet_service.cc:1514] Processing DeleteTablet for tablet aaee7c6830e64fefa378be0078fdfd8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:08 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:36756
I20260620 23:12:08.225885 19587 tablet_service.cc:1514] Processing DeleteTablet for tablet aaee7c6830e64fefa378be0078fdfd8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:08 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:52550
I20260620 23:12:08.226398 19691 tablet_replica.cc:330] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: stopping tablet replica
I20260620 23:12:08.226423 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 19256
I20260620 23:12:08.226500 19691 raft_consensus.cc:2229] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:08.226601 19691 raft_consensus.cc:2258] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:08.226703 19689 tablet_replica.cc:330] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: stopping tablet replica
I20260620 23:12:08.226799 19689 raft_consensus.cc:2229] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:12:08.226948 19689 raft_consensus.cc:2258] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:08.227380 19689 ts_tablet_manager.cc:1905] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:08.228963 19689 ts_tablet_manager.cc:1918] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:08.229039 19689 log.cc:1196] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/aaee7c6830e64fefa378be0078fdfd8e
I20260620 23:12:08.229322 19689 ts_tablet_manager.cc:1939] T aaee7c6830e64fefa378be0078fdfd8e P bef97a69cf4c483e898979af852d15ea: Deleting consensus metadata
I20260620 23:12:08.229684 19178 catalog_manager.cc:4841] TS bef97a69cf4c483e898979af852d15ea (127.1.32.66:45915): tablet aaee7c6830e64fefa378be0078fdfd8e (table test-table [id=ae045be09b2f4f3db461e63490471007]) successfully deleted
I20260620 23:12:08.230046 19691 ts_tablet_manager.cc:1905] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:08.231542 19691 ts_tablet_manager.cc:1918] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:08.231626 19691 log.cc:1196] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/aaee7c6830e64fefa378be0078fdfd8e
I20260620 23:12:08.231904 19691 ts_tablet_manager.cc:1939] T aaee7c6830e64fefa378be0078fdfd8e P 0892b8d71b054a92afc215b1a14bcf15: Deleting consensus metadata
I20260620 23:12:08.232322 19180 catalog_manager.cc:4841] TS 0892b8d71b054a92afc215b1a14bcf15 (127.1.32.67:36099): tablet aaee7c6830e64fefa378be0078fdfd8e (table test-table [id=ae045be09b2f4f3db461e63490471007]) successfully deleted
W20260620 23:12:08.232522 19180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:33393 (error 108)
I20260620 23:12:08.232738 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 19390
I20260620 23:12:08.232990 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.232873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:33393 (local address 127.0.0.1:36758)
0620 23:12:08.232923 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.232953 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:33393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 23:12:08.233085 19180 catalog_manager.cc:4584] TS 345f743a927548f1871a80da099d4af9 (127.1.32.65:33393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aaee7c6830e64fefa378be0078fdfd8e: Network error: Client connection negotiation failed: client connection to 127.1.32.65:33393: connect: Connection refused (error 111)
I20260620 23:12:08.237717 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 19524
I20260620 23:12:08.241994 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 19162
2026-06-20T23:12:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1792 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T23:12:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:08Z Disabled control of system clock
I20260620 23:12:08.268678 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39209
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:39475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:39209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:08.344120 19696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:08.344332 19696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:08.344352 19696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:08.345399 19696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:08.345431 19696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:08.345444 19696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:08.345456 19696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:08.345468 19696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:08.346652 19696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:39209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:39209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:08.346817 19696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:08.347097 19696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:08.351456 19702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:08.351548 19696 server_base.cc:1034] running on GCE node
W20260620 23:12:08.351465 19704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.351488 19701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:08.351984 19696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:08.352252 19696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:08.353420 19696 hybrid_clock.cc:648] HybridClock initialized: now 1781997128353399 us; error 34 us; skew 500 ppm
I20260620 23:12:08.354503 19696 webserver.cc:460] Webserver started at http://127.1.32.126:35617/ using document root <none> and password file <none>
I20260620 23:12:08.354794 19696 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:08.354866 19696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:08.355027 19696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:08.356009 19696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "0900000a2e7245ed8384d913de062029"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "17e98ead18e33fc769dee06b88d34b6c"
server_key_iv: "3bc9eb83bc84580d97300b0a863686e2"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.356351 19696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "0900000a2e7245ed8384d913de062029"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "17e98ead18e33fc769dee06b88d34b6c"
server_key_iv: "3bc9eb83bc84580d97300b0a863686e2"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.357570 19696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:08.358279 19710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.358454 19696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:08.358541 19696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "0900000a2e7245ed8384d913de062029"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "17e98ead18e33fc769dee06b88d34b6c"
server_key_iv: "3bc9eb83bc84580d97300b0a863686e2"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.358623 19696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:08.383985 19696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:08.384258 19696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:08.384415 19696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:08.388204 19696 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:39209
I20260620 23:12:08.388237 19762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:39209 every 8 connection(s)
I20260620 23:12:08.388567 19696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:08.389453 19763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:08.391593 19763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029: Bootstrap starting.
I20260620 23:12:08.392086 19763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:08.392309 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 19696
I20260620 23:12:08.392458 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:08.392467 19763 log.cc:824] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:08.392660 1153 external_mini_cluster.cc:1428] Setting key 3dc3a48732c915ed43f4ca41a2f96146
I20260620 23:12:08.393258 19763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029: No bootstrap required, opened a new log
I20260620 23:12:08.394935 19763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0900000a2e7245ed8384d913de062029" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39209 } }
I20260620 23:12:08.395092 19763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:08.395128 19763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0900000a2e7245ed8384d913de062029, State: Initialized, Role: FOLLOWER
I20260620 23:12:08.395244 19763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0900000a2e7245ed8384d913de062029" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39209 } }
I20260620 23:12:08.395334 19763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:08.395363 19763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:08.395411 19763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:08.396113 19763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0900000a2e7245ed8384d913de062029" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39209 } }
I20260620 23:12:08.396190 19763 leader_election.cc:302] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0900000a2e7245ed8384d913de062029; no voters:
I20260620 23:12:08.396378 19763 leader_election.cc:288] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:08.396404 19768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:08.396564 19768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [term 1 LEADER]: Becoming Leader. State: Replica: 0900000a2e7245ed8384d913de062029, State: Running, Role: LEADER
I20260620 23:12:08.396621 19763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:08.396644 19768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0900000a2e7245ed8384d913de062029" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39209 } }
I20260620 23:12:08.396948 19768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0900000a2e7245ed8384d913de062029. Latest consensus state: current_term: 1 leader_uuid: "0900000a2e7245ed8384d913de062029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0900000a2e7245ed8384d913de062029" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39209 } } }
I20260620 23:12:08.397032 19768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:08.396957 19770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0900000a2e7245ed8384d913de062029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0900000a2e7245ed8384d913de062029" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 39209 } } }
I20260620 23:12:08.397161 19770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029 [sys.catalog]: This master's current role is: LEADER
W20260620 23:12:08.398278 19783 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:08.398339 19783 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:08.398550 19775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:08.398744 19775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:08.400010 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.393200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.1.32.126:39209)
0620 23:12:08.393451 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.393457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.393480 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:12:08.394879 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.394912 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.394982 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.395085 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.395312 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.396243 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.397402 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.397981 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.398087 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.399272 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.399276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.399276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:08.399506 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.399546 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:08.399560 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.399623 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.399723 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.399805 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260620 23:12:08.400135 19775 catalog_manager.cc:1347] Generated new cluster ID: ece80b5b94b3462cb97f206fb8202b95
I20260620 23:12:08.400174 19775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:08.414745 19775 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:08.415232 19775 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:08.423555 19775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029: Generated new TSK 0
I20260620 23:12:08.423717 19775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:08.428406 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39209
--builtin_ntp_servers=127.1.32.84:39475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:08.506500 19787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:08.506713 19787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:08.506748 19787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:08.507988 19787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:08.508073 19787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:08.508114 19787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:12:08.509428 19787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:08.509676 19787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:08.509974 19787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:08.515641 19792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.515595 19795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:08.515820 19787 server_base.cc:1034] running on GCE node
W20260620 23:12:08.515636 19793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:08.516145 19787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:08.516403 19787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:08.517565 19787 hybrid_clock.cc:648] HybridClock initialized: now 1781997128517556 us; error 26 us; skew 500 ppm
I20260620 23:12:08.518752 19787 webserver.cc:460] Webserver started at http://127.1.32.65:44633/ using document root <none> and password file <none>
I20260620 23:12:08.519048 19787 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:08.519094 19787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:08.519255 19787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:08.520267 19787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "331f50a5e31a448383431e6a0ea95917"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "fcdeb00fccd7e2aa419035daa313d129"
server_key_iv: "bb457de0d0f4b1620222a2af178959a7"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.520638 19787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "331f50a5e31a448383431e6a0ea95917"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "fcdeb00fccd7e2aa419035daa313d129"
server_key_iv: "bb457de0d0f4b1620222a2af178959a7"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.521936 19787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:08.522585 19801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.522747 19787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:08.522836 19787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "331f50a5e31a448383431e6a0ea95917"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "fcdeb00fccd7e2aa419035daa313d129"
server_key_iv: "bb457de0d0f4b1620222a2af178959a7"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.522920 19787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:08.536584 19787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:08.536866 19787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:08.537014 19787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:08.537248 19787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:08.537566 19787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:08.537616 19787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.537648 19787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:08.537671 19787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.543596 19787 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:43255
I20260620 23:12:08.543648 19914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:43255 every 8 connection(s)
I20260620 23:12:08.543936 19787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:12:08.549024 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.544597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:49139 (local address 127.1.32.126:39209)
0620 23:12:08.544659 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.544661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.544911 (+ 250us) server_negotiation.cc:407] Connection header received
0620 23:12:08.545417 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.545418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.545433 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.545493 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.546128 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.546637 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.547614 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.547835 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.547918 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.548726 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.548727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.548728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:08.548734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.548747 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:08.548751 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.548834 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.548923 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.548950 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:12:08.549240 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.544567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39209 (local address 127.1.32.65:49139)
0620 23:12:08.544771 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.544803 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.545315 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.545567 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.545571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.545578 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:08.546032 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.546035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.546740 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.546741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.547514 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.547516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.547928 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.547929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.548069 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.548597 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:08.548598 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:08.548624 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:08.548644 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:08.548836 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:08.548842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:08.548845 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.548925 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.549021 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260620 23:12:08.549721 19915 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39209
I20260620 23:12:08.549793 19915 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:08.550006 19915 heartbeater.cc:511] Master 127.1.32.126:39209 requested a full tablet report, sending...
I20260620 23:12:08.550463 19727 ts_manager.cc:194] Registered new tserver with Master: 331f50a5e31a448383431e6a0ea95917 (127.1.32.65:43255)
I20260620 23:12:08.551079 19727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.65:49139
I20260620 23:12:08.552968 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 19787
I20260620 23:12:08.553069 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:12:08.553246 1153 external_mini_cluster.cc:1428] Setting key d6f49a25e6fdc8806bba1ff08939fb03
I20260620 23:12:08.554040 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39209
--builtin_ntp_servers=127.1.32.84:39475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:08.634222 19918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:08.634418 19918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:08.634438 19918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:08.635732 19918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:08.635774 19918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:08.635798 19918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:12:08.637060 19918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:08.637283 19918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:08.637568 19918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:08.642264 19924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.642292 19926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.642264 19923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:08.642747 19918 server_base.cc:1034] running on GCE node
I20260620 23:12:08.643035 19918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:08.643262 19918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:08.644433 19918 hybrid_clock.cc:648] HybridClock initialized: now 1781997128644413 us; error 32 us; skew 500 ppm
I20260620 23:12:08.648444 19918 webserver.cc:460] Webserver started at http://127.1.32.66:40145/ using document root <none> and password file <none>
I20260620 23:12:08.648696 19918 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:08.648762 19918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:08.648902 19918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:08.649966 19918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "42caa9297c224393b47adedbb4ea2a9d"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "663a1d3d8508c4b52d045bc4991766d3"
server_key_iv: "0f901a37d6de7e57becfc5e686876ca0"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.650274 19918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "42caa9297c224393b47adedbb4ea2a9d"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "663a1d3d8508c4b52d045bc4991766d3"
server_key_iv: "0f901a37d6de7e57becfc5e686876ca0"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.651597 19918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:08.652282 19932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.652469 19918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:08.652540 19918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "42caa9297c224393b47adedbb4ea2a9d"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "663a1d3d8508c4b52d045bc4991766d3"
server_key_iv: "0f901a37d6de7e57becfc5e686876ca0"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.652593 19918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:08.681783 19918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:08.682027 19918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:08.682121 19918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:08.682298 19918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:08.682566 19918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:08.682614 19918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.682636 19918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:08.682649 19918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.688225 19918 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:45555
I20260620 23:12:08.688316 20045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:45555 every 8 connection(s)
I20260620 23:12:08.688656 19918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:12:08.693982 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.689371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:36289 (local address 127.1.32.126:39209)
0620 23:12:08.689444 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.689446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.689715 (+ 269us) server_negotiation.cc:407] Connection header received
0620 23:12:08.690271 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.690272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.690283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.690354 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.691059 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.691563 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.692522 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.692729 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.692810 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.693698 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.693699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.693699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:08.693703 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.693717 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:08.693721 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.693764 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.693880 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.693907 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 23:12:08.694183 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.689364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39209 (local address 127.1.32.66:36289)
0620 23:12:08.689575 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.689619 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.690162 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.690421 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.690425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.690434 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:08.690961 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.690966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.691656 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.691658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.692428 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.692430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.692819 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.692820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.692991 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.693540 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:08.693540 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:08.693567 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:08.693588 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:08.693763 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:08.693768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:08.693770 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.693875 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.693971 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260620 23:12:08.694703 20046 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39209
I20260620 23:12:08.694772 20046 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:08.695003 20046 heartbeater.cc:511] Master 127.1.32.126:39209 requested a full tablet report, sending...
I20260620 23:12:08.695348 19727 ts_manager.cc:194] Registered new tserver with Master: 42caa9297c224393b47adedbb4ea2a9d (127.1.32.66:45555)
I20260620 23:12:08.695691 19727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.66:36289
I20260620 23:12:08.698556 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 19918
I20260620 23:12:08.698654 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:12:08.698848 1153 external_mini_cluster.cc:1428] Setting key 4c103717af22ee9f072e71eeb33d4cf9
I20260620 23:12:08.699640 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39209
--builtin_ntp_servers=127.1.32.84:39475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:08.777944 20049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:08.778129 20049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:08.778151 20049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:08.779321 20049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:08.779361 20049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:08.779385 20049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:12:08.780654 20049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:39209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:08.780877 20049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:08.781149 20049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:08.785861 20057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.785876 20055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:08.785907 20054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:08.786266 20049 server_base.cc:1034] running on GCE node
I20260620 23:12:08.786510 20049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:08.786780 20049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:08.787938 20049 hybrid_clock.cc:648] HybridClock initialized: now 1781997128787931 us; error 35 us; skew 500 ppm
I20260620 23:12:08.789121 20049 webserver.cc:460] Webserver started at http://127.1.32.67:33835/ using document root <none> and password file <none>
I20260620 23:12:08.789397 20049 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:08.789470 20049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:08.789626 20049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:08.790632 20049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "173bf8a421d7491385f7ac3996aad7b1"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "10eb1d8ccf74c8b6316daf7bb836f9b9"
server_key_iv: "0cc5e46b2503b5ec618b5210a5880960"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.791050 20049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "173bf8a421d7491385f7ac3996aad7b1"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "10eb1d8ccf74c8b6316daf7bb836f9b9"
server_key_iv: "0cc5e46b2503b5ec618b5210a5880960"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.792362 20049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:12:08.793175 20063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.793383 20049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:12:08.793478 20049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "173bf8a421d7491385f7ac3996aad7b1"
format_stamp: "Formatted at 2026-06-20 23:12:08 on dist-test-slave-fzjm"
server_key: "10eb1d8ccf74c8b6316daf7bb836f9b9"
server_key_iv: "0cc5e46b2503b5ec618b5210a5880960"
server_key_version: "encryptionkey@0"
I20260620 23:12:08.793571 20049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:08.817250 20049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:08.817543 20049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:08.817656 20049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:08.817878 20049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:08.818224 20049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:08.818259 20049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.818320 20049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:08.818351 20049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:08.823895 20049 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:34101
I20260620 23:12:08.823971 20176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:34101 every 8 connection(s)
I20260620 23:12:08.824273 20049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:12:08.829285 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.824985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:37791 (local address 127.1.32.126:39209)
0620 23:12:08.825045 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.825047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.825272 (+ 225us) server_negotiation.cc:407] Connection header received
0620 23:12:08.825835 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.825836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.825849 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.825891 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.826539 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.827021 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.827920 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.828108 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.828203 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.829023 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.829024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.829024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:08.829029 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.829045 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:08.829049 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.829094 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.829183 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.829212 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 23:12:08.829489 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.824978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:39209 (local address 127.1.32.67:37791)
0620 23:12:08.825147 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.825179 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.825727 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.825975 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.825979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.825989 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:08.826452 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.826456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.827115 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.827116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.827825 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.827827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.828210 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.828211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.828364 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.828886 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:08.828887 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:08.828913 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:08.828934 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:08.829094 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:08.829099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:08.829101 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.829185 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.829277 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260620 23:12:08.829905 20177 heartbeater.cc:347] Connected to a master server at 127.1.32.126:39209
I20260620 23:12:08.829982 20177 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:08.830104 20177 heartbeater.cc:511] Master 127.1.32.126:39209 requested a full tablet report, sending...
I20260620 23:12:08.830449 19727 ts_manager.cc:194] Registered new tserver with Master: 173bf8a421d7491385f7ac3996aad7b1 (127.1.32.67:34101)
I20260620 23:12:08.830785 19727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.67:37791
I20260620 23:12:08.834102 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20049
I20260620 23:12:08.834206 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:12:08.834379 1153 external_mini_cluster.cc:1428] Setting key 3ac137a6e55ee29c1b478551921cd393
I20260620 23:12:08.835124 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:12:08.839161 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.836367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.1.32.126:39209)
0620 23:12:08.836406 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.836408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.836518 (+ 110us) server_negotiation.cc:407] Connection header received
0620 23:12:08.836593 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.836593 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.836605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.836657 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:08.836853 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.837279 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.838004 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.838184 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.838260 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.838860 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:08.838861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:08.838861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:08.838865 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:08.838877 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:08.838881 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:08.839009 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.839084 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.839093 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:12:08.840467 19727 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:12:08.841895 19727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:12:08.847718 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.843977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:34101 (local address 127.0.0.1:58454)
0620 23:12:08.844518 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.844526 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.844598 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.844890 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.844891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.844893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.844924 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.844925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.845940 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.845941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.846728 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.846731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.847431 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.847432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.847504 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.847518 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.847580 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.847628 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":63,"threads_started":1}
I20260620 23:12:08.847888 20186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.843807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45555 (local address 127.0.0.1:57866)
0620 23:12:08.843952 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.843989 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.844048 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.844383 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.844385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.844386 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.844491 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.844492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.845500 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.845501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.846689 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.846692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.847573 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.847574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.847632 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.847635 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.847695 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.847725 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":5,"thread_start_us":92,"threads_started":1}
I20260620 23:12:08.847954 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.844036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.1.32.67:34101)
0620 23:12:08.844158 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.844162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.844706 (+ 544us) server_negotiation.cc:407] Connection header received
0620 23:12:08.844741 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.844743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.844756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.844826 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.844997 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.845864 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.846804 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.847340 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.847418 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.847498 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.847836 (+ 338us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.847878 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260620 23:12:08.848073 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.843807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:43255 (local address 127.0.0.1:49724)
0620 23:12:08.844090 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:08.844099 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:08.844146 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:08.844441 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:08.844442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:08.844443 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:08.844491 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.844494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.845706 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.845707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.846693 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:08.846695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.847757 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.847757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:08.847812 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.847816 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:08.847864 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:12:08.847898 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":3,"thread_start_us":45,"threads_started":1}
I20260620 23:12:08.848078 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.843907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.1.32.65:43255)
0620 23:12:08.844276 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.844279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.844288 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:12:08.844334 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.844336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.844349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.844433 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.844592 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.845628 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.846789 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.847495 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.847566 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.847642 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.847947 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.847997 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":84,"threads_started":1}
I20260620 23:12:08.848608 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:08.844086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.1.32.66:45555)
0620 23:12:08.844223 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:08.844228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:08.844238 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:12:08.844276 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:08.844278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:08.844293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:08.844372 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:08.844644 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.845426 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.846789 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:08.847500 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:08.847563 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:08.847642 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:08.847707 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 23:12:08.847746 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260620 23:12:08.848564 20111 tablet_service.cc:1467] Processing CreateTablet for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf (DEFAULT_TABLE table=test-table [id=af5321eb98814efd8d201eea542ef83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:08.848848 19849 tablet_service.cc:1467] Processing CreateTablet for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf (DEFAULT_TABLE table=test-table [id=af5321eb98814efd8d201eea542ef83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:08.848999 20111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:08.849202 19849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:08.849267 19980 tablet_service.cc:1467] Processing CreateTablet for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf (DEFAULT_TABLE table=test-table [id=af5321eb98814efd8d201eea542ef83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:08.849635 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:08.851977 20191 tablet_bootstrap.cc:492] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: Bootstrap starting.
I20260620 23:12:08.852469 20192 tablet_bootstrap.cc:492] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Bootstrap starting.
I20260620 23:12:08.852679 20191 tablet_bootstrap.cc:654] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:08.852953 20193 tablet_bootstrap.cc:492] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Bootstrap starting.
I20260620 23:12:08.853009 20191 log.cc:824] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:08.853061 20192 tablet_bootstrap.cc:654] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:08.853360 20192 log.cc:824] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:08.853698 20193 tablet_bootstrap.cc:654] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:08.853714 20191 tablet_bootstrap.cc:492] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: No bootstrap required, opened a new log
I20260620 23:12:08.853796 20191 ts_tablet_manager.cc:1397] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:08.854027 20193 log.cc:824] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:08.854082 20192 tablet_bootstrap.cc:492] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: No bootstrap required, opened a new log
I20260620 23:12:08.854139 20192 ts_tablet_manager.cc:1397] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:08.854682 20193 tablet_bootstrap.cc:492] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: No bootstrap required, opened a new log
I20260620 23:12:08.854770 20193 ts_tablet_manager.cc:1397] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:08.856307 20191 raft_consensus.cc:348] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:08.856470 20191 raft_consensus.cc:374] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:08.856504 20191 raft_consensus.cc:729] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 331f50a5e31a448383431e6a0ea95917, State: Initialized, Role: FOLLOWER
I20260620 23:12:08.856663 20191 consensus_queue.cc:260] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:08.856922 20191 ts_tablet_manager.cc:1428] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:12:08.856873 20192 raft_consensus.cc:348] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:08.856978 19915 heartbeater.cc:503] Master 127.1.32.126:39209 was elected leader, sending a full tablet report...
I20260620 23:12:08.857002 20192 raft_consensus.cc:374] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:08.857033 20192 raft_consensus.cc:729] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173bf8a421d7491385f7ac3996aad7b1, State: Initialized, Role: FOLLOWER
I20260620 23:12:08.857016 20193 raft_consensus.cc:348] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:08.857162 20193 raft_consensus.cc:374] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:08.857147 20192 consensus_queue.cc:260] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:08.857201 20193 raft_consensus.cc:729] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42caa9297c224393b47adedbb4ea2a9d, State: Initialized, Role: FOLLOWER
I20260620 23:12:08.857333 20193 consensus_queue.cc:260] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:08.857615 20193 ts_tablet_manager.cc:1428] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 23:12:08.857734 20192 ts_tablet_manager.cc:1428] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260620 23:12:08.857872 20177 heartbeater.cc:503] Master 127.1.32.126:39209 was elected leader, sending a full tablet report...
I20260620 23:12:08.858811 20046 heartbeater.cc:503] Master 127.1.32.126:39209 was elected leader, sending a full tablet report...
W20260620 23:12:08.939217 20047 tablet.cc:2369] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:12:09.044579 19916 tablet.cc:2369] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:12:09.074807 20178 tablet.cc:2369] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:12:09.207860 20198 raft_consensus.cc:482] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:09.207971 20198 raft_consensus.cc:504] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:09.208256 20198 leader_election.cc:288] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 331f50a5e31a448383431e6a0ea95917 (127.1.32.65:43255), 42caa9297c224393b47adedbb4ea2a9d (127.1.32.66:45555)
I20260620 23:12:09.212000 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:09.208628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:38645 (local address 127.1.32.65:43255)
0620 23:12:09.208698 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:09.208700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:09.208709 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:12:09.208785 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:09.208785 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:09.208792 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:09.208862 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:09.208979 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.209593 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.211081 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.211615 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.211696 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:09.211705 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:09.211872 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 23:12:09.211907 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 23:12:09.212000 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:09.208560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:56289 (local address 127.1.32.66:45555)
0620 23:12:09.208604 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:09.208606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:09.208614 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:09.208641 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:09.208642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:09.208653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:09.208710 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:09.208868 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.209582 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.211081 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.211615 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.211696 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:09.211705 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:09.211872 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 23:12:09.211906 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:09.212016 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:09.208443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45555 (local address 127.1.32.67:56289)
0620 23:12:09.208487 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:09.208501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:09.208568 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:09.208732 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:09.208733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:09.208735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:09.208802 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:09.208803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.209657 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.209658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:09.210990 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:09.210993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.211707 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.211708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:09.211782 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:09.211785 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:09.211840 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 23:12:09.211892 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":79}
I20260620 23:12:09.212023 20200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:09.208453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:43255 (local address 127.1.32.67:38645)
0620 23:12:09.208661 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:09.208670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:09.208723 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:09.208865 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:09.208865 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:09.208866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:09.208917 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:09.208919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.209657 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.209658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:09.210990 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:09.210992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.211723 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.211723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:09.211782 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:09.211785 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:09.211840 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 23:12:09.211884 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":101,"thread_start_us":100,"threads_started":1}
I20260620 23:12:09.212391 20000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0794aeb3f7cd4ea99bc742e2b7e16bcf" candidate_uuid: "173bf8a421d7491385f7ac3996aad7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42caa9297c224393b47adedbb4ea2a9d" is_pre_election: true
I20260620 23:12:09.212555 20000 raft_consensus.cc:2454] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 173bf8a421d7491385f7ac3996aad7b1 in term 0.
I20260620 23:12:09.212594 19869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0794aeb3f7cd4ea99bc742e2b7e16bcf" candidate_uuid: "173bf8a421d7491385f7ac3996aad7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331f50a5e31a448383431e6a0ea95917" is_pre_election: true
I20260620 23:12:09.212738 19869 raft_consensus.cc:2454] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 173bf8a421d7491385f7ac3996aad7b1 in term 0.
I20260620 23:12:09.212798 20067 leader_election.cc:302] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173bf8a421d7491385f7ac3996aad7b1, 42caa9297c224393b47adedbb4ea2a9d; no voters:
I20260620 23:12:09.212937 20198 raft_consensus.cc:2781] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:12:09.213001 20198 raft_consensus.cc:482] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:12:09.213033 20198 raft_consensus.cc:3037] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:09.213737 20198 raft_consensus.cc:504] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:09.213904 20198 leader_election.cc:288] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [CANDIDATE]: Term 1 election: Requested vote from peers 331f50a5e31a448383431e6a0ea95917 (127.1.32.65:43255), 42caa9297c224393b47adedbb4ea2a9d (127.1.32.66:45555)
I20260620 23:12:09.214061 19869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0794aeb3f7cd4ea99bc742e2b7e16bcf" candidate_uuid: "173bf8a421d7491385f7ac3996aad7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331f50a5e31a448383431e6a0ea95917"
I20260620 23:12:09.214136 19869 raft_consensus.cc:3037] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:09.214850 19869 raft_consensus.cc:2454] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 173bf8a421d7491385f7ac3996aad7b1 in term 1.
I20260620 23:12:09.215003 20000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0794aeb3f7cd4ea99bc742e2b7e16bcf" candidate_uuid: "173bf8a421d7491385f7ac3996aad7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42caa9297c224393b47adedbb4ea2a9d"
I20260620 23:12:09.215050 20065 leader_election.cc:302] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173bf8a421d7491385f7ac3996aad7b1, 331f50a5e31a448383431e6a0ea95917; no voters:
I20260620 23:12:09.215099 20000 raft_consensus.cc:3037] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:09.215224 20198 raft_consensus.cc:2781] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:09.215380 20198 raft_consensus.cc:686] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 1 LEADER]: Becoming Leader. State: Replica: 173bf8a421d7491385f7ac3996aad7b1, State: Running, Role: LEADER
I20260620 23:12:09.215606 20198 consensus_queue.cc:237] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } }
I20260620 23:12:09.215984 20000 raft_consensus.cc:2454] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 173bf8a421d7491385f7ac3996aad7b1 in term 1.
I20260620 23:12:09.216444 19727 catalog_manager.cc:5495] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 173bf8a421d7491385f7ac3996aad7b1 (127.1.32.67). New cstate: current_term: 1 leader_uuid: "173bf8a421d7491385f7ac3996aad7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "173bf8a421d7491385f7ac3996aad7b1" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 34101 } health_report { overall_health: HEALTHY } } }
I20260620 23:12:09.286160 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:09.283962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58466 (local address 127.1.32.67:34101)
0620 23:12:09.284032 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:09.284034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:09.284041 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:09.284076 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:09.284076 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:09.284095 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:09.284155 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:09.284301 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.284758 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.285453 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:09.285624 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:09.285672 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:09.285793 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:09.285874 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:09.285950 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:09.286004 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 23:12:09.286014 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:12:09.287976 19869 raft_consensus.cc:1261] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 1 FOLLOWER]: Refusing update from remote peer 173bf8a421d7491385f7ac3996aad7b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:09.288012 20000 raft_consensus.cc:1261] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 1 FOLLOWER]: Refusing update from remote peer 173bf8a421d7491385f7ac3996aad7b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:09.288327 20198 consensus_queue.cc:1035] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "331f50a5e31a448383431e6a0ea95917" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:09.288484 20201 consensus_queue.cc:1035] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42caa9297c224393b47adedbb4ea2a9d" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:09.290745 20204 mvcc.cc:205] Tried to move back new op lower bound from 7299060241560920064 to 7299060241267724288. Current Snapshot: MvccSnapshot[applied={T|T < 7299060241560920064}]
I20260620 23:12:09.291711 20205 mvcc.cc:205] Tried to move back new op lower bound from 7299060241560920064 to 7299060241267724288. Current Snapshot: MvccSnapshot[applied={T|T < 7299060241560920064 or (T in {7299060241560920064})}]
I20260620 23:12:09.292995 19727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:09.293092 19727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:09.293776 19727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0900000a2e7245ed8384d913de062029: Sending DeleteTablet for 3 replicas of tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf
I20260620 23:12:09.294077 19849 tablet_service.cc:1514] Processing DeleteTablet for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:09 UTC) from {username='slave'} at 127.0.0.1:49724
I20260620 23:12:09.294170 20111 tablet_service.cc:1514] Processing DeleteTablet for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:09 UTC) from {username='slave'} at 127.0.0.1:58454
I20260620 23:12:09.294281 20213 tablet_replica.cc:330] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: stopping tablet replica
I20260620 23:12:09.294308 20214 tablet_replica.cc:330] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: stopping tablet replica
I20260620 23:12:09.294375 20213 raft_consensus.cc:2229] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:09.294384 20214 raft_consensus.cc:2229] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:12:09.294435 19980 tablet_service.cc:1514] Processing DeleteTablet for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:09 UTC) from {username='slave'} at 127.0.0.1:57866
I20260620 23:12:09.294464 20214 raft_consensus.cc:2258] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:09.294471 20213 raft_consensus.cc:2258] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:09.294865 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 19787
I20260620 23:12:09.294902 20213 ts_tablet_manager.cc:1905] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 331f50a5e31a448383431e6a0ea95917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:09.294958 20214 ts_tablet_manager.cc:1905] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:09.299005 20215 tablet_replica.cc:330] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: stopping tablet replica
I20260620 23:12:09.299100 20215 raft_consensus.cc:2229] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:09.299193 20215 raft_consensus.cc:2258] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:09.299304 20214 ts_tablet_manager.cc:1918] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:09.299373 20214 log.cc:1196] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/0794aeb3f7cd4ea99bc742e2b7e16bcf
I20260620 23:12:09.299600 20215 ts_tablet_manager.cc:1905] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:09.299613 20214 ts_tablet_manager.cc:1939] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 173bf8a421d7491385f7ac3996aad7b1: Deleting consensus metadata
I20260620 23:12:09.299968 19712 catalog_manager.cc:4841] TS 173bf8a421d7491385f7ac3996aad7b1 (127.1.32.67:34101): tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf (table test-table [id=af5321eb98814efd8d201eea542ef83d]) successfully deleted
W20260620 23:12:09.300745 19712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:43255 (error 108)
I20260620 23:12:09.301043 20215 ts_tablet_manager.cc:1918] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:09.301107 20215 log.cc:1196] T 0794aeb3f7cd4ea99bc742e2b7e16bcf P 42caa9297c224393b47adedbb4ea2a9d: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/0794aeb3f7cd4ea99bc742e2b7e16bcf
I20260620 23:12:09.301210 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 19918
I20260620 23:12:09.301242 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:09.301133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:43255 (local address 127.0.0.1:49726)
0620 23:12:09.301176 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:09.301213 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:43255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 23:12:09.301735 19712 catalog_manager.cc:4584] TS 331f50a5e31a448383431e6a0ea95917 (127.1.32.65:43255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0794aeb3f7cd4ea99bc742e2b7e16bcf: Network error: Client connection negotiation failed: client connection to 127.1.32.65:43255: connect: Connection refused (error 111)
I20260620 23:12:09.306200 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:09.306120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45555 (local address 127.0.0.1:57882)
0620 23:12:09.306153 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:09.306175 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:45555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260620 23:12:09.306490 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20049
I20260620 23:12:09.310632 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 19696
2026-06-20T23:12:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1067 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:09 dist-test-slave-fzjm krb5kdc[20219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:09 dist-test-slave-fzjm krb5kdc[20219](info): set up 2 sockets
Jun 20 23:12:09 dist-test-slave-fzjm krb5kdc[20219](info): commencing operation
krb5kdc: starting...
W20260620 23:12:10.318746 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:10.425742 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43941
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:43941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:10.501650 20232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:10.501901 20232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:10.501940 20232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:10.503137 20232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:10.503202 20232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:10.503226 20232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:10.503253 20232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:10.503283 20232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:10.504717 20232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:43941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:10.504974 20232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:10.505190 20232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:10.509827 20238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:10.509855 20237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:10.510116 20232 server_base.cc:1034] running on GCE node
W20260620 23:12:10.509866 20240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:10.510389 20232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:10.510684 20232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:10.511873 20232 hybrid_clock.cc:648] HybridClock initialized: now 1781997130511857 us; error 27 us; skew 500 ppm
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:10.514365 20232 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:10.514812 20232 webserver.cc:460] Webserver started at http://127.1.32.126:39673/ using document root <none> and password file <none>
I20260620 23:12:10.515060 20232 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:10.515130 20232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:10.515275 20232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:10.516289 20232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "e76cc08560054d549a692b8b7c3135ae"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "830af80881742733da42b6c1a5e5aca1"
server_key_iv: "a246030fea91655af1d88c0ef7f6aa7f"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.516637 20232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "e76cc08560054d549a692b8b7c3135ae"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "830af80881742733da42b6c1a5e5aca1"
server_key_iv: "a246030fea91655af1d88c0ef7f6aa7f"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.517935 20232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:10.518646 20247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.518811 20232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:10.518922 20232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "e76cc08560054d549a692b8b7c3135ae"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "830af80881742733da42b6c1a5e5aca1"
server_key_iv: "a246030fea91655af1d88c0ef7f6aa7f"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.519012 20232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:10.534708 20232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:10.536468 20232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:10.536618 20232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:10.540385 20232 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:43941
I20260620 23:12:10.540421 20299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:43941 every 8 connection(s)
I20260620 23:12:10.540743 20232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:10.541570 20300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:10.543854 20300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae: Bootstrap starting.
I20260620 23:12:10.544415 20300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:10.544695 20300 log.cc:824] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:10.545341 20300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae: No bootstrap required, opened a new log
I20260620 23:12:10.547312 20300 raft_consensus.cc:348] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e76cc08560054d549a692b8b7c3135ae" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43941 } }
I20260620 23:12:10.547487 20300 raft_consensus.cc:374] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:10.547509 20300 raft_consensus.cc:729] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e76cc08560054d549a692b8b7c3135ae, State: Initialized, Role: FOLLOWER
I20260620 23:12:10.547658 20300 consensus_queue.cc:260] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e76cc08560054d549a692b8b7c3135ae" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43941 } }
I20260620 23:12:10.547727 20300 raft_consensus.cc:388] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:10.547747 20300 raft_consensus.cc:482] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:10.547808 20300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:10.548345 20300 raft_consensus.cc:504] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e76cc08560054d549a692b8b7c3135ae" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43941 } }
I20260620 23:12:10.548456 20300 leader_election.cc:302] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e76cc08560054d549a692b8b7c3135ae; no voters:
I20260620 23:12:10.548604 20300 leader_election.cc:288] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:10.548627 20303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:10.548794 20303 raft_consensus.cc:686] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [term 1 LEADER]: Becoming Leader. State: Replica: e76cc08560054d549a692b8b7c3135ae, State: Running, Role: LEADER
I20260620 23:12:10.548892 20300 sys_catalog.cc:564] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:10.548877 20303 consensus_queue.cc:237] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e76cc08560054d549a692b8b7c3135ae" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43941 } }
I20260620 23:12:10.549643 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20232
I20260620 23:12:10.549780 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:10.550009 1153 external_mini_cluster.cc:1428] Setting key a920d222ab5e0d19f0689ceb8fcf868b
I20260620 23:12:10.550333 20305 sys_catalog.cc:455] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader e76cc08560054d549a692b8b7c3135ae. Latest consensus state: current_term: 1 leader_uuid: "e76cc08560054d549a692b8b7c3135ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e76cc08560054d549a692b8b7c3135ae" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43941 } } }
I20260620 23:12:10.550405 20304 sys_catalog.cc:455] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e76cc08560054d549a692b8b7c3135ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e76cc08560054d549a692b8b7c3135ae" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43941 } } }
I20260620 23:12:10.550776 20305 sys_catalog.cc:458] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:10.550781 20304 sys_catalog.cc:458] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae [sys.catalog]: This master's current role is: LEADER
W20260620 23:12:10.550838 20318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:10.550910 20318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:10.551008 20322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:10.551198 20322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:10.552587 20322 catalog_manager.cc:1347] Generated new cluster ID: 11a7c64a788547cd8be0dbebe6caf498
I20260620 23:12:10.552635 20322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:10.562542 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:10.550473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.1.32.126:43941)
0620 23:12:10.550989 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:10.550998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:10.551030 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:12:10.551143 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:10.551150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:10.551205 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:10.551340 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:10.552079 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.553403 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.554545 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.555189 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.555579 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:10.557961 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:10.557965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:10.557965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:10.557976 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:10.560514 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:10.560907 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:10.560910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:10.560911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:10.561038 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:10.561269 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:10.561270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:10.561270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:10.561574 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:10.562122 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:10.562191 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 23:12:10.562323 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":99,"threads_started":1}
I20260620 23:12:10.566939 20322 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:10.567736 20322 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:10.573937 20322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae: Generated new TSK 0
I20260620 23:12:10.574098 20322 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.32.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
Entry for principal kudu/127.1.32.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab.
I20260620 23:12:10.601846 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43941
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:10.681965 20326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:10.682153 20326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:10.682178 20326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:10.683357 20326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:10.683398 20326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:10.683423 20326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:12:10.684849 20326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.65.keytab
--principal=kudu/127.1.32.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:10.685075 20326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:10.685292 20326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:10.690064 20332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:10.690073 20331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:10.690088 20326 server_base.cc:1034] running on GCE node
W20260620 23:12:10.690073 20334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:10.690469 20326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:10.690721 20326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:10.691860 20326 hybrid_clock.cc:648] HybridClock initialized: now 1781997130691839 us; error 42 us; skew 500 ppm
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:10.694263 20326 init.cc:377] Logged in from keytab as kudu/127.1.32.65@KRBTEST.COM (short username kudu)
I20260620 23:12:10.694623 20326 webserver.cc:460] Webserver started at http://127.1.32.65:38851/ using document root <none> and password file <none>
I20260620 23:12:10.694864 20326 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:10.694970 20326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:10.695103 20326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:10.696071 20326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "45fe43ed84174b2f887402d8b35e6770"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "e46fbda3ff396b787b60878a53a83db9"
server_key_iv: "d4fb43fedd39b993e78e9eef2cbd86e6"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.696435 20326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "45fe43ed84174b2f887402d8b35e6770"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "e46fbda3ff396b787b60878a53a83db9"
server_key_iv: "d4fb43fedd39b993e78e9eef2cbd86e6"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.697678 20326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:10.698339 20341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.698491 20326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:10.698577 20326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "45fe43ed84174b2f887402d8b35e6770"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "e46fbda3ff396b787b60878a53a83db9"
server_key_iv: "d4fb43fedd39b993e78e9eef2cbd86e6"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.698647 20326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:10.714175 20326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:10.716091 20326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:10.716241 20326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:10.716477 20326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:10.716769 20326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:10.716826 20326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.716866 20326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:10.716892 20326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.722294 20326 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:42279
I20260620 23:12:10.722361 20454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:42279 every 8 connection(s)
I20260620 23:12:10.722674 20326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:12:10.726195 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20326
I20260620 23:12:10.726276 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:12:10.726466 1153 external_mini_cluster.cc:1428] Setting key ce459789d5134152514aada079821793
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.65@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:10.732481 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:10.723373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:56703 (local address 127.1.32.126:43941)
0620 23:12:10.723418 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:10.723420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:10.723633 (+ 213us) server_negotiation.cc:407] Connection header received
0620 23:12:10.724128 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:10.724129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:10.724143 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:10.724199 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:10.725119 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.725763 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.726947 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.727223 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.727292 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:10.729664 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:10.729666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:10.729666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:10.729674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:10.731150 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:10.731536 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:10.731537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:10.731538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:10.731597 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:10.731859 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:10.731860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:10.731860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:10.731972 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:10.732042 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:10.732314 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 23:12:10.732394 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:12:10.732669 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:10.723358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:43941 (local address 127.1.32.65:56703)
0620 23:12:10.723513 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:10.723546 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:10.724028 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:10.724265 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:10.724269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:10.724829 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:10.725001 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:10.725006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.725852 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.725853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:10.726843 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:10.726845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.727300 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.727301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:10.727441 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:10.727926 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:10.727942 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:10.729546 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:10.731242 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:10.731250 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:10.731252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:10.731461 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:10.731663 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:10.731664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:10.731664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:10.731783 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:10.732048 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:10.732055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:10.732182 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 23:12:10.732286 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:12:10.732431 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260620 23:12:10.733136 20455 heartbeater.cc:347] Connected to a master server at 127.1.32.126:43941
I20260620 23:12:10.733222 20455 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:10.733409 20455 heartbeater.cc:511] Master 127.1.32.126:43941 requested a full tablet report, sending...
I20260620 23:12:10.733914 20264 ts_manager.cc:194] Registered new tserver with Master: 45fe43ed84174b2f887402d8b35e6770 (127.1.32.65:42279)
WARNING: no policy specified for kudu/127.1.32.66@KRBTEST.COM; defaulting to no policy
I20260620 23:12:10.734555 20264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.65@KRBTEST.COM'} at 127.1.32.65:56703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
Entry for principal kudu/127.1.32.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab.
I20260620 23:12:10.749783 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43941
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:10.829715 20460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:10.829900 20460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:10.829938 20460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:10.831032 20460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:10.831067 20460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:10.831094 20460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:12:10.832484 20460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.66.keytab
--principal=kudu/127.1.32.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:10.832659 20460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:10.832811 20460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:10.837522 20466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:10.837558 20460 server_base.cc:1034] running on GCE node
W20260620 23:12:10.837577 20465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:10.837538 20468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:10.837891 20460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:10.838152 20460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:10.839287 20460 hybrid_clock.cc:648] HybridClock initialized: now 1781997130839271 us; error 36 us; skew 500 ppm
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:10.841671 20460 init.cc:377] Logged in from keytab as kudu/127.1.32.66@KRBTEST.COM (short username kudu)
I20260620 23:12:10.842123 20460 webserver.cc:460] Webserver started at http://127.1.32.66:41031/ using document root <none> and password file <none>
I20260620 23:12:10.842315 20460 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:10.842355 20460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:10.842437 20460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:10.843429 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "fcc83bb323894562a396d4fa2281b43f"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "f4b838f6a67859f935bd7de7eb92db0a"
server_key_iv: "35c083f4e6e8e4f87378e5f4ad3f2386"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.843716 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "fcc83bb323894562a396d4fa2281b43f"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "f4b838f6a67859f935bd7de7eb92db0a"
server_key_iv: "35c083f4e6e8e4f87378e5f4ad3f2386"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.845281 20460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:10.846052 20475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.846256 20460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:10.846350 20460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "fcc83bb323894562a396d4fa2281b43f"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "f4b838f6a67859f935bd7de7eb92db0a"
server_key_iv: "35c083f4e6e8e4f87378e5f4ad3f2386"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.846433 20460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:10.855067 20460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:10.856680 20460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:10.856822 20460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:10.857064 20460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:10.857370 20460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:10.857422 20460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.857460 20460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:10.857486 20460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.862802 20460 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:45385
I20260620 23:12:10.862848 20588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:45385 every 8 connection(s)
I20260620 23:12:10.863158 20460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:12:10.864149 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20460
I20260620 23:12:10.864253 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:12:10.864434 1153 external_mini_cluster.cc:1428] Setting key de9212dc8c5273d31f9757cdc1b8f120
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.66@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:10.872653 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:10.863829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:39607 (local address 127.1.32.126:43941)
0620 23:12:10.863895 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:10.863897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:10.864106 (+ 209us) server_negotiation.cc:407] Connection header received
0620 23:12:10.864733 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:10.864734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:10.864744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:10.864803 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:10.865745 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.866476 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.867470 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.867766 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.867836 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:10.869854 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:10.869855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:10.869856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:10.869863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:10.871317 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:10.871678 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:10.871679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:10.871680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:10.871752 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:10.872032 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:10.872033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:10.872034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:10.872175 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:10.872255 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:10.872482 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 23:12:10.872566 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:12:10.872680 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:10.863833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:43941 (local address 127.1.32.66:39607)
0620 23:12:10.863983 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:10.864026 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:10.864625 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:10.864866 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:10.864869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:10.865482 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:10.865648 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:10.865653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.866560 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.866561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:10.867376 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:10.867378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:10.867870 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:10.867871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:10.867968 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:10.868433 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:10.868441 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:10.869763 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:10.871390 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:10.871395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:10.871396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:10.871590 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:10.871825 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:10.871825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:10.871826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:10.871960 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:10.872249 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:10.872253 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:10.872356 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 23:12:10.872435 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 23:12:10.872541 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.1.32.67@KRBTEST.COM; defaulting to no policy
I20260620 23:12:10.873229 20589 heartbeater.cc:347] Connected to a master server at 127.1.32.126:43941
I20260620 23:12:10.873302 20589 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:10.873488 20589 heartbeater.cc:511] Master 127.1.32.126:43941 requested a full tablet report, sending...
I20260620 23:12:10.873888 20264 ts_manager.cc:194] Registered new tserver with Master: fcc83bb323894562a396d4fa2281b43f (127.1.32.66:45385)
I20260620 23:12:10.874282 20264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.66@KRBTEST.COM'} at 127.1.32.66:39607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
Entry for principal kudu/127.1.32.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab.
I20260620 23:12:10.887822 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43941
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:10.967242 20594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:10.967460 20594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:10.967494 20594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:10.968570 20594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:10.968642 20594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:10.968688 20594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:12:10.970046 20594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.67.keytab
--principal=kudu/127.1.32.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:43941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:10.970309 20594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:10.970530 20594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:10.975304 20599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:10.975427 20600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:10.975303 20602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:10.975469 20594 server_base.cc:1034] running on GCE node
I20260620 23:12:10.975653 20594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:10.975876 20594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:10.977034 20594 hybrid_clock.cc:648] HybridClock initialized: now 1781997130977029 us; error 37 us; skew 500 ppm
Jun 20 23:12:10 dist-test-slave-fzjm krb5kdc[20219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:10.979548 20594 init.cc:377] Logged in from keytab as kudu/127.1.32.67@KRBTEST.COM (short username kudu)
I20260620 23:12:10.979969 20594 webserver.cc:460] Webserver started at http://127.1.32.67:37183/ using document root <none> and password file <none>
I20260620 23:12:10.980175 20594 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:10.980237 20594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:10.980386 20594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:10.981316 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "984d60ae5dc441c6988e606072c6c73b"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "10abcd20dbb335c58a5c46142374f05e"
server_key_iv: "e65f796b19c06416adf288f9cbfef588"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.981659 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "984d60ae5dc441c6988e606072c6c73b"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "10abcd20dbb335c58a5c46142374f05e"
server_key_iv: "e65f796b19c06416adf288f9cbfef588"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.982998 20594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:12:10.983750 20609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.983912 20594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:10.984001 20594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "984d60ae5dc441c6988e606072c6c73b"
format_stamp: "Formatted at 2026-06-20 23:12:10 on dist-test-slave-fzjm"
server_key: "10abcd20dbb335c58a5c46142374f05e"
server_key_iv: "e65f796b19c06416adf288f9cbfef588"
server_key_version: "encryptionkey@0"
I20260620 23:12:10.984088 20594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:10.994361 20594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:10.996477 20594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:10.996590 20594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:10.996737 20594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:10.996958 20594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:10.996990 20594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:10.997010 20594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:10.997022 20594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.002713 20594 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:38999
I20260620 23:12:11.002764 20722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:38999 every 8 connection(s)
I20260620 23:12:11.003106 20594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
Jun 20 23:12:11 dist-test-slave-fzjm krb5kdc[20219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.67@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:11.011722 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.003828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:43215 (local address 127.1.32.126:43941)
0620 23:12:11.003900 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.003903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.004243 (+ 340us) server_negotiation.cc:407] Connection header received
0620 23:12:11.004662 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.004663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.004673 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.004724 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:11.005587 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.006055 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.007023 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.007216 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.007264 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.009201 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:11.009202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:11.009203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:11.009210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:11.010461 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:11.010813 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:11.010814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:11.010814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:11.010861 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:11.011229 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:11.011230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:11.011230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:11.011338 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:11.011387 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.011610 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.011665 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 23:12:11.011925 20725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.003783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:43941 (local address 127.1.32.67:43215)
0620 23:12:11.003972 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.004124 (+ 152us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.004580 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.004817 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.004820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.005336 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:11.005510 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.005514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.006126 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.006126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.006882 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.006941 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.007279 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.007280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.007397 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.007830 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:11.007838 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:11.009115 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:11.010531 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:11.010537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:11.010538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:11.010741 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:11.011078 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:11.011079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:11.011079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:11.011172 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:11.011395 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:11.011399 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:11.011500 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.011584 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.011685 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260620 23:12:11.012389 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20594
I20260620 23:12:11.012480 20723 heartbeater.cc:347] Connected to a master server at 127.1.32.126:43941
I20260620 23:12:11.012503 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:12:11.012570 20723 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:11.012696 1153 external_mini_cluster.cc:1428] Setting key 3a81e70af1991fefa0766c3e095eda74
I20260620 23:12:11.012758 20723 heartbeater.cc:511] Master 127.1.32.126:43941 requested a full tablet report, sending...
I20260620 23:12:11.013123 20263 ts_manager.cc:194] Registered new tserver with Master: 984d60ae5dc441c6988e606072c6c73b (127.1.32.67:38999)
I20260620 23:12:11.013505 20263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.32.67@KRBTEST.COM'} at 127.1.32.67:43215
I20260620 23:12:11.013587 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:12:11.020493 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.014852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.1.32.126:43941)
0620 23:12:11.014915 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.014917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.015063 (+ 146us) server_negotiation.cc:407] Connection header received
0620 23:12:11.015117 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.015118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.015127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.015169 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:11.015707 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.016141 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.016888 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.017072 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.017132 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.018248 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:11.018249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:11.018249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:11.018254 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:11.019546 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:11.019864 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:11.019864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:11.019865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:11.019913 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:11.020070 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:11.020070 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:11.020071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:11.020181 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:11.020249 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.020349 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.020402 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:12:11.022338 20263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:12:11.024025 20263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:12:11.030543 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.026515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.1.32.67:38999)
0620 23:12:11.026691 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.026694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.026703 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:12:11.026738 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.026740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.026753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.026839 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.027020 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.028180 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.029661 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.030127 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.030213 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.030270 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.030427 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.030466 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260620 23:12:11.030535 20733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.026102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:38999 (local address 127.0.0.1:36378)
0620 23:12:11.026423 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.026431 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.026471 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.026866 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.026866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.026868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.026949 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.026950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.028273 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.028274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.029564 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.029566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.030213 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.030214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.030293 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.030306 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.030408 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.030462 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":32,"thread_start_us":63,"threads_started":1}
I20260620 23:12:11.030633 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.026102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42279 (local address 127.0.0.1:57562)
0620 23:12:11.026352 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.026392 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.026462 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.027241 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.027241 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.027243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.027278 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.027280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.028448 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.028449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.029547 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.029551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.030433 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.030434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.030489 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.030493 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.030522 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.030562 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":2,"thread_start_us":270,"threads_started":1}
I20260620 23:12:11.030915 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.026348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.1.32.65:42279)
0620 23:12:11.027098 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.027103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.027113 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:12:11.027147 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.027149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.027163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.027244 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.027344 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.028364 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.029673 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.030340 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.030698 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.030761 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.030784 (+ 23us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.030820 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":73,"threads_started":1}
I20260620 23:12:11.031543 20389 tablet_service.cc:1467] Processing CreateTablet for tablet e1505a8cbaaa457da3affe5f8ec8a8cb (DEFAULT_TABLE table=test-table [id=f4494a29368e4aab866651c8883a0573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:11.031543 20657 tablet_service.cc:1467] Processing CreateTablet for tablet e1505a8cbaaa457da3affe5f8ec8a8cb (DEFAULT_TABLE table=test-table [id=f4494a29368e4aab866651c8883a0573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:11.031881 20657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1505a8cbaaa457da3affe5f8ec8a8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:11.031881 20389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1505a8cbaaa457da3affe5f8ec8a8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:11.035190 20738 tablet_bootstrap.cc:492] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Bootstrap starting.
I20260620 23:12:11.035826 20738 tablet_bootstrap.cc:654] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:11.035848 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.026164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45385 (local address 127.0.0.1:57678)
0620 23:12:11.026499 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.026511 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.026567 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.026829 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.026833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.026836 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.026921 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.026923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.029415 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.029416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.030470 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.030472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.035585 (+ 5113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.035586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.035648 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.035652 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.035734 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.035777 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260620 23:12:11.035871 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.026285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57678 (local address 127.1.32.66:45385)
0620 23:12:11.026635 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.026641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.026653 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:12:11.026694 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.026696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.026715 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.026801 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.028505 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.029318 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.030553 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.035489 (+ 4936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.035581 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.035635 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.035750 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.035796 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":117,"threads_started":1}
I20260620 23:12:11.036381 20738 log.cc:824] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:11.036442 20523 tablet_service.cc:1467] Processing CreateTablet for tablet e1505a8cbaaa457da3affe5f8ec8a8cb (DEFAULT_TABLE table=test-table [id=f4494a29368e4aab866651c8883a0573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:11.036733 20737 tablet_bootstrap.cc:492] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770: Bootstrap starting.
I20260620 23:12:11.036785 20523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1505a8cbaaa457da3affe5f8ec8a8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:11.037118 20738 tablet_bootstrap.cc:492] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: No bootstrap required, opened a new log
I20260620 23:12:11.037199 20738 ts_tablet_manager.cc:1397] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:11.038715 20738 raft_consensus.cc:348] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.038960 20738 raft_consensus.cc:374] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:11.039032 20738 raft_consensus.cc:729] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984d60ae5dc441c6988e606072c6c73b, State: Initialized, Role: FOLLOWER
I20260620 23:12:11.039162 20738 consensus_queue.cc:260] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.039462 20723 heartbeater.cc:503] Master 127.1.32.126:43941 was elected leader, sending a full tablet report...
I20260620 23:12:11.039583 20737 tablet_bootstrap.cc:654] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:11.039784 20741 tablet_bootstrap.cc:492] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: Bootstrap starting.
I20260620 23:12:11.040177 20738 ts_tablet_manager.cc:1428] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:12:11.040254 20737 log.cc:824] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:11.040527 20741 tablet_bootstrap.cc:654] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:11.041096 20737 tablet_bootstrap.cc:492] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770: No bootstrap required, opened a new log
I20260620 23:12:11.041090 20741 log.cc:824] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:11.041189 20737 ts_tablet_manager.cc:1397] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 23:12:11.041862 20741 tablet_bootstrap.cc:492] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: No bootstrap required, opened a new log
I20260620 23:12:11.041911 20741 ts_tablet_manager.cc:1397] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:11.042815 20737 raft_consensus.cc:348] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.043015 20737 raft_consensus.cc:374] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:11.043064 20737 raft_consensus.cc:729] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fe43ed84174b2f887402d8b35e6770, State: Initialized, Role: FOLLOWER
I20260620 23:12:11.043179 20737 consensus_queue.cc:260] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.043450 20455 heartbeater.cc:503] Master 127.1.32.126:43941 was elected leader, sending a full tablet report...
I20260620 23:12:11.043450 20737 ts_tablet_manager.cc:1428] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:11.043393 20741 raft_consensus.cc:348] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.043533 20741 raft_consensus.cc:374] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:11.043555 20741 raft_consensus.cc:729] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc83bb323894562a396d4fa2281b43f, State: Initialized, Role: FOLLOWER
I20260620 23:12:11.043655 20741 consensus_queue.cc:260] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.043921 20741 ts_tablet_manager.cc:1428] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 23:12:11.043934 20589 heartbeater.cc:503] Master 127.1.32.126:43941 was elected leader, sending a full tablet report...
W20260620 23:12:11.113848 20590 tablet.cc:2369] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:12:11.153339 20745 raft_consensus.cc:482] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:11.153463 20745 raft_consensus.cc:504] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.153770 20745 leader_election.cc:288] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 984d60ae5dc441c6988e606072c6c73b (127.1.32.67:38999), 45fe43ed84174b2f887402d8b35e6770 (127.1.32.65:42279)
I20260620 23:12:11.156782 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.154120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:37331 (local address 127.1.32.67:38999)
0620 23:12:11.154205 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.154208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.154242 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:12:11.154335 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.154336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.154348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.154408 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.154592 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.155062 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.156012 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.156422 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.156518 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.156526 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.156673 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.156711 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 23:12:11.156795 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.154138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:41311 (local address 127.1.32.65:42279)
0620 23:12:11.154197 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.154199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.154210 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:12:11.154241 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.154242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.154255 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.154313 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.154498 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.155075 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.156071 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.156425 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.156477 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.156510 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.156689 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.156726 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 23:12:11.156801 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.153993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42279 (local address 127.1.32.66:41311)
0620 23:12:11.154066 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.154081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.154143 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.154333 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.154334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.154335 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.154413 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.154416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.155173 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.155174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.155951 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.155953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.156511 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.156512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.156597 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.156600 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.156677 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.156713 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":34}
I20260620 23:12:11.156807 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.153993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:38999 (local address 127.1.32.66:37331)
0620 23:12:11.154179 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.154190 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.154255 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.154424 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.154425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.154426 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.154517 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.154518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.155136 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.155137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.155920 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.155922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.156538 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.156538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.156597 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.156601 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.156666 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.156707 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":18,"thread_start_us":94,"threads_started":1}
I20260620 23:12:11.157176 20409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1505a8cbaaa457da3affe5f8ec8a8cb" candidate_uuid: "fcc83bb323894562a396d4fa2281b43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fe43ed84174b2f887402d8b35e6770" is_pre_election: true
I20260620 23:12:11.157212 20677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1505a8cbaaa457da3affe5f8ec8a8cb" candidate_uuid: "fcc83bb323894562a396d4fa2281b43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984d60ae5dc441c6988e606072c6c73b" is_pre_election: true
I20260620 23:12:11.157321 20409 raft_consensus.cc:2454] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcc83bb323894562a396d4fa2281b43f in term 0.
I20260620 23:12:11.157401 20677 raft_consensus.cc:2454] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcc83bb323894562a396d4fa2281b43f in term 0.
I20260620 23:12:11.157529 20479 leader_election.cc:302] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45fe43ed84174b2f887402d8b35e6770, fcc83bb323894562a396d4fa2281b43f; no voters:
I20260620 23:12:11.157680 20745 raft_consensus.cc:2781] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:12:11.157745 20745 raft_consensus.cc:482] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:12:11.157789 20745 raft_consensus.cc:3037] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:11.158480 20745 raft_consensus.cc:504] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.158624 20745 leader_election.cc:288] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [CANDIDATE]: Term 1 election: Requested vote from peers 984d60ae5dc441c6988e606072c6c73b (127.1.32.67:38999), 45fe43ed84174b2f887402d8b35e6770 (127.1.32.65:42279)
I20260620 23:12:11.158759 20409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1505a8cbaaa457da3affe5f8ec8a8cb" candidate_uuid: "fcc83bb323894562a396d4fa2281b43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fe43ed84174b2f887402d8b35e6770"
I20260620 23:12:11.158778 20677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1505a8cbaaa457da3affe5f8ec8a8cb" candidate_uuid: "fcc83bb323894562a396d4fa2281b43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984d60ae5dc441c6988e606072c6c73b"
I20260620 23:12:11.158852 20409 raft_consensus.cc:3037] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:11.158840 20677 raft_consensus.cc:3037] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:11.159530 20677 raft_consensus.cc:2454] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcc83bb323894562a396d4fa2281b43f in term 1.
I20260620 23:12:11.159644 20409 raft_consensus.cc:2454] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcc83bb323894562a396d4fa2281b43f in term 1.
I20260620 23:12:11.159703 20476 leader_election.cc:302] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984d60ae5dc441c6988e606072c6c73b, fcc83bb323894562a396d4fa2281b43f; no voters:
I20260620 23:12:11.159811 20745 raft_consensus.cc:2781] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:11.159981 20745 raft_consensus.cc:686] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 1 LEADER]: Becoming Leader. State: Replica: fcc83bb323894562a396d4fa2281b43f, State: Running, Role: LEADER
I20260620 23:12:11.160076 20745 consensus_queue.cc:237] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } }
I20260620 23:12:11.160881 20264 catalog_manager.cc:5495] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f reported cstate change: term changed from 0 to 1, leader changed from <none> to fcc83bb323894562a396d4fa2281b43f (127.1.32.66). New cstate: current_term: 1 leader_uuid: "fcc83bb323894562a396d4fa2281b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcc83bb323894562a396d4fa2281b43f" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 45385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 } health_report { overall_health: UNKNOWN } } }
I20260620 23:12:11.177210 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.174953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.1.32.66:45385)
0620 23:12:11.174997 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.174999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.175007 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:11.175106 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.175107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.175131 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.175204 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:11.175325 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.175765 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.176500 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.176681 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.176752 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.176918 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:11.177007 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:11.177095 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.177115 (+ 20us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.177135 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 23:12:11.178931 20409 raft_consensus.cc:1261] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 1 FOLLOWER]: Refusing update from remote peer fcc83bb323894562a396d4fa2281b43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:11.178932 20677 raft_consensus.cc:1261] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 1 FOLLOWER]: Refusing update from remote peer fcc83bb323894562a396d4fa2281b43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:11.179287 20745 consensus_queue.cc:1035] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "984d60ae5dc441c6988e606072c6c73b" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 38999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:11.179420 20747 consensus_queue.cc:1035] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fe43ed84174b2f887402d8b35e6770" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 42279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:11.182291 20751 mvcc.cc:205] Tried to move back new op lower bound from 7299060249306132480 to 7299060249232236544. Current Snapshot: MvccSnapshot[applied={T|T < 7299060249306132480 or (T in {7299060249306132480})}]
I20260620 23:12:11.182657 20752 mvcc.cc:205] Tried to move back new op lower bound from 7299060249306132480 to 7299060249232236544. Current Snapshot: MvccSnapshot[applied={T|T < 7299060249306132480}]
I20260620 23:12:11.183676 20263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:11.183787 20263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:11.184500 20263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e76cc08560054d549a692b8b7c3135ae: Sending DeleteTablet for 3 replicas of tablet e1505a8cbaaa457da3affe5f8ec8a8cb
I20260620 23:12:11.184845 20523 tablet_service.cc:1514] Processing DeleteTablet for tablet e1505a8cbaaa457da3affe5f8ec8a8cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:11 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:57678
I20260620 23:12:11.184875 20657 tablet_service.cc:1514] Processing DeleteTablet for tablet e1505a8cbaaa457da3affe5f8ec8a8cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:11 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:36378
I20260620 23:12:11.185046 20389 tablet_service.cc:1514] Processing DeleteTablet for tablet e1505a8cbaaa457da3affe5f8ec8a8cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:11 UTC) from {username='kudu', principal='kudu/127.1.32.126@KRBTEST.COM'} at 127.0.0.1:57562
I20260620 23:12:11.185161 20760 tablet_replica.cc:330] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: stopping tablet replica
I20260620 23:12:11.185315 20760 raft_consensus.cc:2229] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:11.185352 20761 tablet_replica.cc:330] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770: stopping tablet replica
I20260620 23:12:11.185434 20761 raft_consensus.cc:2229] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:11.185437 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20326
I20260620 23:12:11.185525 20761 raft_consensus.cc:2258] T e1505a8cbaaa457da3affe5f8ec8a8cb P 45fe43ed84174b2f887402d8b35e6770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:11.185539 20760 raft_consensus.cc:2258] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:11.186743 20760 ts_tablet_manager.cc:1905] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:11.188723 20760 ts_tablet_manager.cc:1918] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:11.188800 20760 log.cc:1196] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/e1505a8cbaaa457da3affe5f8ec8a8cb
I20260620 23:12:11.189080 20760 ts_tablet_manager.cc:1939] T e1505a8cbaaa457da3affe5f8ec8a8cb P 984d60ae5dc441c6988e606072c6c73b: Deleting consensus metadata
I20260620 23:12:11.189484 20248 catalog_manager.cc:4841] TS 984d60ae5dc441c6988e606072c6c73b (127.1.32.67:38999): tablet e1505a8cbaaa457da3affe5f8ec8a8cb (table test-table [id=f4494a29368e4aab866651c8883a0573]) successfully deleted
W20260620 23:12:11.190851 20251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:42279 (error 108)
I20260620 23:12:11.185263 20759 tablet_replica.cc:330] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f: stopping tablet replica
I20260620 23:12:11.191004 20759 raft_consensus.cc:2229] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:12:11.191094 20759 raft_consensus.cc:2258] T e1505a8cbaaa457da3affe5f8ec8a8cb P fcc83bb323894562a396d4fa2281b43f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:11.191419 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20460
I20260620 23:12:11.191427 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.191263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:42279 (local address 127.0.0.1:57566)
0620 23:12:11.191326 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.191368 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:42279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 23:12:11.193799 20251 catalog_manager.cc:4584] TS 45fe43ed84174b2f887402d8b35e6770 (127.1.32.65:42279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1505a8cbaaa457da3affe5f8ec8a8cb: Network error: Client connection negotiation failed: client connection to 127.1.32.65:42279: connect: Connection refused (error 111)
I20260620 23:12:11.197104 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20594
I20260620 23:12:11.197130 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.196979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:45385 (local address 127.0.0.1:57700)
0620 23:12:11.197059 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.197093 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.66:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 23:12:11.201319 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20232
2026-06-20T23:12:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1891 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T23:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:11Z Disabled control of system clock
I20260620 23:12:11.228122 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:42797
--webserver_interface=127.1.32.126
--webserver_port=0
--builtin_ntp_servers=127.1.32.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:42797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:11.302973 20766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:11.303179 20766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:11.303198 20766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:11.304275 20766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:11.304311 20766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:11.304324 20766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:11.304335 20766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:11.304355 20766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:11.305572 20766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:42797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:42797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:11.305748 20766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:11.305970 20766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:11.310410 20771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:11.310421 20772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:11.310441 20766 server_base.cc:1034] running on GCE node
W20260620 23:12:11.310390 20774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:11.310937 20766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:11.311249 20766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:11.312398 20766 hybrid_clock.cc:648] HybridClock initialized: now 1781997131312384 us; error 31 us; skew 500 ppm
I20260620 23:12:11.313628 20766 webserver.cc:460] Webserver started at http://127.1.32.126:43147/ using document root <none> and password file <none>
I20260620 23:12:11.313815 20766 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:11.313862 20766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:11.313946 20766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:11.314945 20766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "8ecb61b1a21040eb94d4a83246c9cc57"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "8bf3b161f0c5418377cd31bad36acbf3"
server_key_iv: "9e5bcdc268d00aaf88fa4ba586c6fbcf"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.315249 20766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "8ecb61b1a21040eb94d4a83246c9cc57"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "8bf3b161f0c5418377cd31bad36acbf3"
server_key_iv: "9e5bcdc268d00aaf88fa4ba586c6fbcf"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.316385 20766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:12:11.317029 20780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.317188 20766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:11.317286 20766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "8ecb61b1a21040eb94d4a83246c9cc57"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "8bf3b161f0c5418377cd31bad36acbf3"
server_key_iv: "9e5bcdc268d00aaf88fa4ba586c6fbcf"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.317378 20766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:11.339608 20766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:11.339912 20766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:11.340056 20766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:11.344184 20766 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:42797
I20260620 23:12:11.344220 20832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:42797 every 8 connection(s)
I20260620 23:12:11.344574 20766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:11.345523 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:11.347712 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57: Bootstrap starting.
I20260620 23:12:11.348234 20833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:11.348471 20833 log.cc:824] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:11.349049 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57: No bootstrap required, opened a new log
I20260620 23:12:11.350482 20833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42797 } }
I20260620 23:12:11.350606 20833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:11.350628 20833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ecb61b1a21040eb94d4a83246c9cc57, State: Initialized, Role: FOLLOWER
I20260620 23:12:11.350755 20833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42797 } }
I20260620 23:12:11.350833 20833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:11.350857 20833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:11.350919 20833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:11.351408 20833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42797 } }
I20260620 23:12:11.351501 20833 leader_election.cc:302] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ecb61b1a21040eb94d4a83246c9cc57; no voters:
I20260620 23:12:11.351624 20833 leader_election.cc:288] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:11.351712 20836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:11.351871 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20766
I20260620 23:12:11.351908 20836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [term 1 LEADER]: Becoming Leader. State: Replica: 8ecb61b1a21040eb94d4a83246c9cc57, State: Running, Role: LEADER
I20260620 23:12:11.351895 20833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:11.351992 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:11.352065 20836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42797 } }
I20260620 23:12:11.352221 1153 external_mini_cluster.cc:1428] Setting key a1d99b4bdaef6ba95de71b90f940e1d9
I20260620 23:12:11.356448 20836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ecb61b1a21040eb94d4a83246c9cc57. Latest consensus state: current_term: 1 leader_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42797 } } }
I20260620 23:12:11.356563 20836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:11.356632 20837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecb61b1a21040eb94d4a83246c9cc57" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 42797 } } }
I20260620 23:12:11.356706 20837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57 [sys.catalog]: This master's current role is: LEADER
W20260620 23:12:11.356962 20852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:11.357033 20852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:11.357132 20855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:11.357335 20855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:11.358703 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.352886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47612 (local address 127.1.32.126:42797)
0620 23:12:11.353603 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.353611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.353639 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:12:11.353744 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.353750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.353814 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.354160 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:11.354251 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.355549 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.356653 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.357251 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.357367 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.358039 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:11.358042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:11.358043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:11.358263 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:11.358300 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:11.358319 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:11.358396 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.358467 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.358539 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":94,"threads_started":1}
I20260620 23:12:11.359241 20855 catalog_manager.cc:1347] Generated new cluster ID: 685a064dbb0d4adc95b2c0dc97d6e875
I20260620 23:12:11.359295 20855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:11.379303 20855 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:11.379868 20855 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:11.396222 20855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57: Generated new TSK 0
I20260620 23:12:11.396411 20855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:11.405009 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.65:0
--local_ip_for_outbound_sockets=127.1.32.65
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42797
--builtin_ntp_servers=127.1.32.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:11.481855 20857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:11.482066 20857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:11.482101 20857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:11.483299 20857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:11.483364 20857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:11.483407 20857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.65
I20260620 23:12:11.484706 20857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.32.65
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.65
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:11.484961 20857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:11.485234 20857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:11.489828 20865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:11.489912 20857 server_base.cc:1034] running on GCE node
W20260620 23:12:11.489835 20862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:11.489861 20863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:11.490226 20857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:11.490473 20857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:11.491637 20857 hybrid_clock.cc:648] HybridClock initialized: now 1781997131491614 us; error 44 us; skew 500 ppm
I20260620 23:12:11.492760 20857 webserver.cc:460] Webserver started at http://127.1.32.65:43291/ using document root <none> and password file <none>
I20260620 23:12:11.492995 20857 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:11.493059 20857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:11.493204 20857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:11.494163 20857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data/instance:
uuid: "04b6a032e8e34bdfbc95fe9c77524dc3"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "e418fbf4fc5b335d72f514e522547d37"
server_key_iv: "6fccd51b068ef5c7d01ed2ffe262f3b5"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.494516 20857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance:
uuid: "04b6a032e8e34bdfbc95fe9c77524dc3"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "e418fbf4fc5b335d72f514e522547d37"
server_key_iv: "6fccd51b068ef5c7d01ed2ffe262f3b5"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.495820 20857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:11.496507 20871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.496675 20857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:11.496766 20857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/wal
uuid: "04b6a032e8e34bdfbc95fe9c77524dc3"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "e418fbf4fc5b335d72f514e522547d37"
server_key_iv: "6fccd51b068ef5c7d01ed2ffe262f3b5"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.496847 20857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:11.523308 20857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:11.523605 20857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:11.523746 20857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:11.523963 20857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:11.524271 20857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:11.524372 20857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.524415 20857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:11.524456 20857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.530124 20857 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.65:34893
I20260620 23:12:11.530169 20984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.65:34893 every 8 connection(s)
I20260620 23:12:11.530484 20857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/data/info.pb
I20260620 23:12:11.535404 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.531228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:43801 (local address 127.1.32.126:42797)
0620 23:12:11.531278 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.531280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.531484 (+ 204us) server_negotiation.cc:407] Connection header received
0620 23:12:11.531945 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.531946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.531958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.532028 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:11.532654 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.533150 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.534072 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.534275 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.534339 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.535170 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:11.535171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:11.535172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:11.535176 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:11.535190 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:11.535194 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:11.535241 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.535319 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.535344 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:12:11.535596 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.531185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42797 (local address 127.1.32.65:43801)
0620 23:12:11.531363 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.531407 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.531863 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.532081 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.532084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.532092 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:11.532572 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.532577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.533230 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.533231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.533978 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.533980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.534380 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.534381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.534482 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.535026 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:11.535026 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:11.535069 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:11.535089 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:11.535250 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:11.535254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:11.535256 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.535304 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.535386 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260620 23:12:11.536162 20985 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42797
I20260620 23:12:11.536271 20985 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:11.536456 20985 heartbeater.cc:511] Master 127.1.32.126:42797 requested a full tablet report, sending...
I20260620 23:12:11.536940 20797 ts_manager.cc:194] Registered new tserver with Master: 04b6a032e8e34bdfbc95fe9c77524dc3 (127.1.32.65:34893)
I20260620 23:12:11.537500 20797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.65:43801
I20260620 23:12:11.539790 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20857
I20260620 23:12:11.539924 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-0/wal/instance
I20260620 23:12:11.540109 1153 external_mini_cluster.cc:1428] Setting key ce32d1ded671197758df3ecf087e571d
I20260620 23:12:11.540974 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.66:0
--local_ip_for_outbound_sockets=127.1.32.66
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42797
--builtin_ntp_servers=127.1.32.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:11.617632 20988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:11.617852 20988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:11.617877 20988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:11.619000 20988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:11.619041 20988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:11.619067 20988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.66
I20260620 23:12:11.620538 20988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.32.66
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.66
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:11.620798 20988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:11.621099 20988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:11.626004 20994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:11.626011 20996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:11.626035 20993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:11.626223 20988 server_base.cc:1034] running on GCE node
I20260620 23:12:11.626421 20988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:11.626659 20988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:11.627826 20988 hybrid_clock.cc:648] HybridClock initialized: now 1781997131627805 us; error 30 us; skew 500 ppm
I20260620 23:12:11.629055 20988 webserver.cc:460] Webserver started at http://127.1.32.66:40055/ using document root <none> and password file <none>
I20260620 23:12:11.629299 20988 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:11.629374 20988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:11.629516 20988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:11.630502 20988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data/instance:
uuid: "5370973b11fc4722a35521dd59d2ef41"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "42a096cf44ca51114d43538dfdff2c34"
server_key_iv: "c2e816eecd237b6c24257bc6c42b24cd"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.630856 20988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance:
uuid: "5370973b11fc4722a35521dd59d2ef41"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "42a096cf44ca51114d43538dfdff2c34"
server_key_iv: "c2e816eecd237b6c24257bc6c42b24cd"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.632087 20988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:11.632726 21002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.632876 20988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:11.632965 20988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal
uuid: "5370973b11fc4722a35521dd59d2ef41"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "42a096cf44ca51114d43538dfdff2c34"
server_key_iv: "c2e816eecd237b6c24257bc6c42b24cd"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.633038 20988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:11.649639 20988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:11.649906 20988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:11.650053 20988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:11.650252 20988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:11.650552 20988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:11.650604 20988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.650641 20988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:11.650671 20988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.656678 20988 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.66:46447
I20260620 23:12:11.656738 21115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.66:46447 every 8 connection(s)
I20260620 23:12:11.657014 20988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/data/info.pb
I20260620 23:12:11.661577 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.657664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.66:46119 (local address 127.1.32.126:42797)
0620 23:12:11.657735 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.657738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.657969 (+ 231us) server_negotiation.cc:407] Connection header received
0620 23:12:11.658372 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.658373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.658382 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.658426 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:11.659004 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.659464 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.660336 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.660499 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.660543 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.661332 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:11.661333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:11.661334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:11.661337 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:11.661351 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:11.661356 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:11.661405 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.661489 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.661511 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:12:11.661767 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.657659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42797 (local address 127.1.32.66:46119)
0620 23:12:11.657857 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.657886 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.658282 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.658479 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.658482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.658490 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:11.658927 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.658931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.659539 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.659539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.660256 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.660258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.660563 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.660563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.660668 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.661169 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:11.661170 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:11.661196 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:11.661217 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:11.661421 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:11.661425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:11.661427 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.661480 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.661561 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260620 23:12:11.662374 21116 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42797
I20260620 23:12:11.662465 21116 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:11.662621 21116 heartbeater.cc:511] Master 127.1.32.126:42797 requested a full tablet report, sending...
I20260620 23:12:11.663004 20797 ts_manager.cc:194] Registered new tserver with Master: 5370973b11fc4722a35521dd59d2ef41 (127.1.32.66:46447)
I20260620 23:12:11.663335 20797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.66:46119
I20260620 23:12:11.665216 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 20988
I20260620 23:12:11.665308 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal/instance
I20260620 23:12:11.665445 1153 external_mini_cluster.cc:1428] Setting key 688abce56ee07b3b676979a7d7d5061e
I20260620 23:12:11.666364 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.32.67:0
--local_ip_for_outbound_sockets=127.1.32.67
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42797
--builtin_ntp_servers=127.1.32.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:12:11.744391 21119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:11.744570 21119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:11.744591 21119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:11.745679 21119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:11.745718 21119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:11.745741 21119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.32.67
I20260620 23:12:11.747123 21119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.32.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.32.67
--webserver_port=0
--tserver_master_addrs=127.1.32.126:42797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.32.67
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:11.747344 21119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:11.747617 21119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:11.752368 21127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:11.752395 21124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:11.752408 21125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:11.752583 21119 server_base.cc:1034] running on GCE node
I20260620 23:12:11.752813 21119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:11.753052 21119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:11.754206 21119 hybrid_clock.cc:648] HybridClock initialized: now 1781997131754193 us; error 28 us; skew 500 ppm
I20260620 23:12:11.755450 21119 webserver.cc:460] Webserver started at http://127.1.32.67:36687/ using document root <none> and password file <none>
I20260620 23:12:11.755679 21119 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:11.755746 21119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:11.755887 21119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:11.756872 21119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data/instance:
uuid: "ded4c9861c9942bd8b6bcb30457bffb2"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "c1edc33a2602028a3731ff7e28595f24"
server_key_iv: "ddbd297670d64baf25e19f58abb07881"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.757230 21119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance:
uuid: "ded4c9861c9942bd8b6bcb30457bffb2"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "c1edc33a2602028a3731ff7e28595f24"
server_key_iv: "ddbd297670d64baf25e19f58abb07881"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.758433 21119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 23:12:11.759176 21133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.759343 21119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.759434 21119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal
uuid: "ded4c9861c9942bd8b6bcb30457bffb2"
format_stamp: "Formatted at 2026-06-20 23:12:11 on dist-test-slave-fzjm"
server_key: "c1edc33a2602028a3731ff7e28595f24"
server_key_iv: "ddbd297670d64baf25e19f58abb07881"
server_key_version: "encryptionkey@0"
I20260620 23:12:11.759510 21119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:11.780298 21119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:11.780551 21119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:11.780689 21119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:11.780944 21119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:12:11.781243 21119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:12:11.781296 21119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.781327 21119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:12:11.781355 21119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:11.786731 21119 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.67:41399
I20260620 23:12:11.786764 21246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.67:41399 every 8 connection(s)
I20260620 23:12:11.787097 21119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/data/info.pb
I20260620 23:12:11.790975 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21119
I20260620 23:12:11.791079 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal/instance
I20260620 23:12:11.791260 1153 external_mini_cluster.cc:1428] Setting key ebc7e9100c2828a01d1bd5540273750e
I20260620 23:12:11.792109 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.787858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.67:38345 (local address 127.1.32.126:42797)
0620 23:12:11.787912 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.787914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.788126 (+ 212us) server_negotiation.cc:407] Connection header received
0620 23:12:11.788546 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.788546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.788555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.788610 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:11.789205 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.789728 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.790628 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.790811 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.790859 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.791856 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:11.791857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:11.791857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:11.791876 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:11.791890 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:11.791895 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:11.791944 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.792021 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.792046 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:12:11.792338 21249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.787784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:42797 (local address 127.1.32.67:38345)
0620 23:12:11.787993 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.788029 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.788472 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.788678 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.788681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.788688 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:11.789140 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.789144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.789838 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.789839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.790559 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.790561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.790992 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.790993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.791143 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.791687 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:12:11.791688 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:11.791722 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:12:11.791747 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:11.791956 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:11.791961 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:11.791964 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.792026 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.792120 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
I20260620 23:12:11.792668 21247 heartbeater.cc:347] Connected to a master server at 127.1.32.126:42797
I20260620 23:12:11.792762 21247 heartbeater.cc:464] Registering TS with master...
I20260620 23:12:11.792919 21247 heartbeater.cc:511] Master 127.1.32.126:42797 requested a full tablet report, sending...
I20260620 23:12:11.793227 20797 ts_manager.cc:194] Registered new tserver with Master: ded4c9861c9942bd8b6bcb30457bffb2 (127.1.32.67:41399)
I20260620 23:12:11.793578 20797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.32.67:38345
I20260620 23:12:11.803159 1153 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:12:11.807049 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.804507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47626 (local address 127.1.32.126:42797)
0620 23:12:11.804558 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.804560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.804674 (+ 114us) server_negotiation.cc:407] Connection header received
0620 23:12:11.804696 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.804697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.804708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.804771 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:11.804907 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.805349 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.806055 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.806236 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.806298 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.806822 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:11.806823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:11.806823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:11.806827 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:11.806838 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:11.806842 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:11.806927 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.806968 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.806997 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 23:12:11.808421 20797 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:12:11.809885 20797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:12:11.816226 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.812066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:41399 (local address 127.0.0.1:37416)
0620 23:12:11.812228 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.812266 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.812345 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.812681 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.812684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.812687 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.812751 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.812753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.813972 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.813973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.815090 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.815093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.815921 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.815922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.816011 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.816026 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.816093 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.816141 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":42,"thread_start_us":101,"threads_started":1}
I20260620 23:12:11.816236 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.812320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.1.32.67:41399)
0620 23:12:11.812507 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.812510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.812517 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:11.812545 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.812547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.812558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.812669 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.812822 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.813890 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.815169 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.815839 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.815903 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.816005 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.816106 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.816140 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260620 23:12:11.816334 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.812228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.1.32.65:34893)
0620 23:12:11.812427 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.812432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.813187 (+ 755us) server_negotiation.cc:407] Connection header received
0620 23:12:11.813255 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.813257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.813273 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.813360 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.813489 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.814451 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.815486 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.816014 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.816088 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.816174 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.816205 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.816244 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260620 23:12:11.816706 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.812067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:34893 (local address 127.0.0.1:49030)
0620 23:12:11.813102 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.813111 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.813183 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.813373 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.813374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.813376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.813410 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.813411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.814534 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.814535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.815404 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.815406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.816087 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.816089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.816147 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.816151 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.816195 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.816233 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260620 23:12:11.816712 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.812317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.1.32.66:46447)
0620 23:12:11.812496 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.812499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.812881 (+ 382us) server_negotiation.cc:407] Connection header received
0620 23:12:11.812930 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.812932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.812947 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.813033 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.813169 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.813974 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.815201 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.815831 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.815903 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.816005 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.816595 (+ 590us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.816637 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260620 23:12:11.817003 21181 tablet_service.cc:1467] Processing CreateTablet for tablet c3efc60a6296442994fce830f0616eaa (DEFAULT_TABLE table=test-table [id=4b34bf376b4946198bd0bcde196839a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:11.817342 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3efc60a6296442994fce830f0616eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:11.819078 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.812067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:46447 (local address 127.0.0.1:45200)
0620 23:12:11.812754 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.812764 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.812812 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.813039 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.813040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.813043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.813082 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.813083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.814040 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.814041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.815099 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.815102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.816435 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.816436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.816499 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.816502 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.816580 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.818998 (+ 2418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":22,"thread_start_us":65,"threads_started":1}
I20260620 23:12:11.819576 20918 tablet_service.cc:1467] Processing CreateTablet for tablet c3efc60a6296442994fce830f0616eaa (DEFAULT_TABLE table=test-table [id=4b34bf376b4946198bd0bcde196839a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:11.819746 21050 tablet_service.cc:1467] Processing CreateTablet for tablet c3efc60a6296442994fce830f0616eaa (DEFAULT_TABLE table=test-table [id=4b34bf376b4946198bd0bcde196839a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:12:11.819952 20918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3efc60a6296442994fce830f0616eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:11.820029 21050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3efc60a6296442994fce830f0616eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:11.820178 21261 tablet_bootstrap.cc:492] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Bootstrap starting.
I20260620 23:12:11.820825 21261 tablet_bootstrap.cc:654] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:11.821121 21261 log.cc:824] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:11.821764 21261 tablet_bootstrap.cc:492] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: No bootstrap required, opened a new log
I20260620 23:12:11.821851 21261 ts_tablet_manager.cc:1397] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:11.823196 21264 tablet_bootstrap.cc:492] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Bootstrap starting.
I20260620 23:12:11.824043 21264 tablet_bootstrap.cc:654] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:11.824378 21264 log.cc:824] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:11.824347 21261 raft_consensus.cc:348] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.824502 21261 raft_consensus.cc:374] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:11.824551 21261 raft_consensus.cc:729] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded4c9861c9942bd8b6bcb30457bffb2, State: Initialized, Role: FOLLOWER
I20260620 23:12:11.824695 21263 tablet_bootstrap.cc:492] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3: Bootstrap starting.
I20260620 23:12:11.824690 21261 consensus_queue.cc:260] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.824955 21261 ts_tablet_manager.cc:1428] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:12:11.825001 21247 heartbeater.cc:503] Master 127.1.32.126:42797 was elected leader, sending a full tablet report...
I20260620 23:12:11.825284 21263 tablet_bootstrap.cc:654] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:11.825554 21263 log.cc:824] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:11.825999 21264 tablet_bootstrap.cc:492] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: No bootstrap required, opened a new log
I20260620 23:12:11.826088 21264 ts_tablet_manager.cc:1397] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 23:12:11.826316 21263 tablet_bootstrap.cc:492] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3: No bootstrap required, opened a new log
I20260620 23:12:11.826396 21263 ts_tablet_manager.cc:1397] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 23:12:11.828052 21264 raft_consensus.cc:348] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.828168 21264 raft_consensus.cc:374] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:11.828187 21264 raft_consensus.cc:729] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5370973b11fc4722a35521dd59d2ef41, State: Initialized, Role: FOLLOWER
I20260620 23:12:11.828298 21264 consensus_queue.cc:260] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.828420 21263 raft_consensus.cc:348] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.828534 21264 ts_tablet_manager.cc:1428] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 23:12:11.828581 21263 raft_consensus.cc:374] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:11.828594 21116 heartbeater.cc:503] Master 127.1.32.126:42797 was elected leader, sending a full tablet report...
I20260620 23:12:11.828624 21263 raft_consensus.cc:729] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b6a032e8e34bdfbc95fe9c77524dc3, State: Initialized, Role: FOLLOWER
I20260620 23:12:11.828734 21263 consensus_queue.cc:260] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.828966 21263 ts_tablet_manager.cc:1428] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 23:12:11.829085 20985 heartbeater.cc:503] Master 127.1.32.126:42797 was elected leader, sending a full tablet report...
I20260620 23:12:11.873168 21269 raft_consensus.cc:482] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:12:11.873304 21269 raft_consensus.cc:504] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.873744 21269 leader_election.cc:288] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ded4c9861c9942bd8b6bcb30457bffb2 (127.1.32.67:41399), 5370973b11fc4722a35521dd59d2ef41 (127.1.32.66:46447)
I20260620 23:12:11.877483 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.874044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:51469 (local address 127.1.32.67:41399)
0620 23:12:11.874134 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.874137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.874146 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:12:11.874176 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.874177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.874187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.874240 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.874415 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.875206 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.876575 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.877112 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.877194 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.877204 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.877375 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.877412 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:12:11.877483 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.874018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.32.65:48013 (local address 127.1.32.66:46447)
0620 23:12:11.874076 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.874077 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.874198 (+ 121us) server_negotiation.cc:407] Connection header received
0620 23:12:11.874285 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.874286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.874296 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.874323 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:12:11.874654 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.875351 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.876575 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.877124 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.877194 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.877204 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.877375 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.877414 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 23:12:11.877529 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.873927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.67:41399 (local address 127.1.32.65:51469)
0620 23:12:11.873986 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.873998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.874053 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.874257 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.874258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.874259 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.874331 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.874333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.875294 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.875295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.876456 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.876459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.877206 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.877207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.877285 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.877288 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.877349 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.877416 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":6}
I20260620 23:12:11.877535 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.873934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.66:46447 (local address 127.1.32.65:48013)
0620 23:12:11.874129 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.874137 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:11.874209 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:11.874427 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:11.874428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:11.874430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:12:11.874565 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.874567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.875416 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.875416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.876469 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:11.876470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.877207 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.877208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:11.877285 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.877289 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 23:12:11.877350 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 23:12:11.877411 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":31,"thread_start_us":111,"threads_started":1}
I20260620 23:12:11.877880 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3efc60a6296442994fce830f0616eaa" candidate_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" is_pre_election: true
I20260620 23:12:11.877910 21070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3efc60a6296442994fce830f0616eaa" candidate_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5370973b11fc4722a35521dd59d2ef41" is_pre_election: true
I20260620 23:12:11.878021 21201 raft_consensus.cc:2454] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04b6a032e8e34bdfbc95fe9c77524dc3 in term 0.
I20260620 23:12:11.878069 21070 raft_consensus.cc:2454] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04b6a032e8e34bdfbc95fe9c77524dc3 in term 0.
I20260620 23:12:11.878250 20874 leader_election.cc:302] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b6a032e8e34bdfbc95fe9c77524dc3, ded4c9861c9942bd8b6bcb30457bffb2; no voters:
I20260620 23:12:11.878435 21269 raft_consensus.cc:2781] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:12:11.878494 21269 raft_consensus.cc:482] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:12:11.878520 21269 raft_consensus.cc:3037] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:11.879262 21269 raft_consensus.cc:504] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.879412 21269 leader_election.cc:288] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [CANDIDATE]: Term 1 election: Requested vote from peers ded4c9861c9942bd8b6bcb30457bffb2 (127.1.32.67:41399), 5370973b11fc4722a35521dd59d2ef41 (127.1.32.66:46447)
I20260620 23:12:11.879550 21201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3efc60a6296442994fce830f0616eaa" candidate_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ded4c9861c9942bd8b6bcb30457bffb2"
I20260620 23:12:11.879611 21070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3efc60a6296442994fce830f0616eaa" candidate_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5370973b11fc4722a35521dd59d2ef41"
I20260620 23:12:11.879652 21201 raft_consensus.cc:3037] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:11.879717 21070 raft_consensus.cc:3037] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:11.880370 21201 raft_consensus.cc:2454] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04b6a032e8e34bdfbc95fe9c77524dc3 in term 1.
I20260620 23:12:11.880447 21070 raft_consensus.cc:2454] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04b6a032e8e34bdfbc95fe9c77524dc3 in term 1.
I20260620 23:12:11.880570 20874 leader_election.cc:302] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b6a032e8e34bdfbc95fe9c77524dc3, ded4c9861c9942bd8b6bcb30457bffb2; no voters:
I20260620 23:12:11.880692 21269 raft_consensus.cc:2781] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:11.880828 21269 raft_consensus.cc:686] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [term 1 LEADER]: Becoming Leader. State: Replica: 04b6a032e8e34bdfbc95fe9c77524dc3, State: Running, Role: LEADER
I20260620 23:12:11.880928 21269 consensus_queue.cc:237] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } }
I20260620 23:12:11.881757 20797 catalog_manager.cc:5495] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04b6a032e8e34bdfbc95fe9c77524dc3 (127.1.32.65). New cstate: current_term: 1 leader_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b6a032e8e34bdfbc95fe9c77524dc3" member_type: VOTER last_known_addr { host: "127.1.32.65" port: 34893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 } health_report { overall_health: UNKNOWN } } }
I20260620 23:12:11.893415 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.891216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.1.32.65:34893)
0620 23:12:11.891286 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:11.891288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:11.891296 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:11.891321 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:11.891322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:11.891336 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:11.891397 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:11.891495 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.891910 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.892584 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:11.892759 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:11.892823 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:11.892935 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:11.893022 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:11.893086 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:11.893181 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 23:12:11.893198 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:12:11.895144 21201 raft_consensus.cc:1261] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 1 FOLLOWER]: Refusing update from remote peer 04b6a032e8e34bdfbc95fe9c77524dc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:11.895469 21070 raft_consensus.cc:1261] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 1 FOLLOWER]: Refusing update from remote peer 04b6a032e8e34bdfbc95fe9c77524dc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:11.895475 21269 consensus_queue.cc:1035] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ded4c9861c9942bd8b6bcb30457bffb2" member_type: VOTER last_known_addr { host: "127.1.32.67" port: 41399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:11.895715 21269 consensus_queue.cc:1035] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5370973b11fc4722a35521dd59d2ef41" member_type: VOTER last_known_addr { host: "127.1.32.66" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:11.898087 21274 mvcc.cc:205] Tried to move back new op lower bound from 7299060252239683584 to 7299060252184956928. Current Snapshot: MvccSnapshot[applied={T|T < 7299060252239683584}]
I20260620 23:12:11.898406 21275 mvcc.cc:205] Tried to move back new op lower bound from 7299060252239683584 to 7299060252184956928. Current Snapshot: MvccSnapshot[applied={T|T < 7299060252239683584}]
I20260620 23:12:11.898695 21277 mvcc.cc:205] Tried to move back new op lower bound from 7299060252239683584 to 7299060252184956928. Current Snapshot: MvccSnapshot[applied={T|T < 7299060252239683584}]
I20260620 23:12:11.900439 20795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:11.900558 20795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:12:11.901356 20795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ecb61b1a21040eb94d4a83246c9cc57: Sending DeleteTablet for 3 replicas of tablet c3efc60a6296442994fce830f0616eaa
I20260620 23:12:11.901692 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet c3efc60a6296442994fce830f0616eaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:11 UTC) from {username='slave'} at 127.0.0.1:37416
I20260620 23:12:11.901711 20918 tablet_service.cc:1514] Processing DeleteTablet for tablet c3efc60a6296442994fce830f0616eaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:11 UTC) from {username='slave'} at 127.0.0.1:49030
I20260620 23:12:11.901726 21050 tablet_service.cc:1514] Processing DeleteTablet for tablet c3efc60a6296442994fce830f0616eaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:12:11 UTC) from {username='slave'} at 127.0.0.1:45200
I20260620 23:12:11.901945 21285 tablet_replica.cc:330] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: stopping tablet replica
I20260620 23:12:11.901961 21284 tablet_replica.cc:330] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: stopping tablet replica
I20260620 23:12:11.902048 21284 raft_consensus.cc:2229] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:11.902132 21286 tablet_replica.cc:330] T c3efc60a6296442994fce830f0616eaa P 04b6a032e8e34bdfbc95fe9c77524dc3: stopping tablet replica
I20260620 23:12:11.902139 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20857
I20260620 23:12:11.902216 21284 raft_consensus.cc:2258] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:11.903074 21285 raft_consensus.cc:2229] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:12:11.903227 21284 ts_tablet_manager.cc:1905] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:11.903246 21285 raft_consensus.cc:2258] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:12:11.903739 21285 ts_tablet_manager.cc:1905] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:12:11.904801 21284 ts_tablet_manager.cc:1918] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:11.904883 21284 log.cc:1196] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-2/wal/wals/c3efc60a6296442994fce830f0616eaa
I20260620 23:12:11.905192 21284 ts_tablet_manager.cc:1939] T c3efc60a6296442994fce830f0616eaa P ded4c9861c9942bd8b6bcb30457bffb2: Deleting consensus metadata
I20260620 23:12:11.905426 21285 ts_tablet_manager.cc:1918] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:12:11.905504 21285 log.cc:1196] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Deleting WAL directory at /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996897900784-1153-0/minicluster-data/ts-1/wal/wals/c3efc60a6296442994fce830f0616eaa
I20260620 23:12:11.905653 20783 catalog_manager.cc:4841] TS ded4c9861c9942bd8b6bcb30457bffb2 (127.1.32.67:41399): tablet c3efc60a6296442994fce830f0616eaa (table test-table [id=4b34bf376b4946198bd0bcde196839a3]) successfully deleted
I20260620 23:12:11.905778 21285 ts_tablet_manager.cc:1939] T c3efc60a6296442994fce830f0616eaa P 5370973b11fc4722a35521dd59d2ef41: Deleting consensus metadata
I20260620 23:12:11.906139 20782 catalog_manager.cc:4841] TS 5370973b11fc4722a35521dd59d2ef41 (127.1.32.66:46447): tablet c3efc60a6296442994fce830f0616eaa (table test-table [id=4b34bf376b4946198bd0bcde196839a3]) successfully deleted
W20260620 23:12:11.907790 20784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.32.65:34893 (error 108)
I20260620 23:12:11.907982 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20988
I20260620 23:12:11.909062 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:11.908116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.65:34893 (local address 127.0.0.1:49060)
0620 23:12:11.908978 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:11.909025 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.65:34893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":839}
W20260620 23:12:11.909186 20784 catalog_manager.cc:4584] TS 04b6a032e8e34bdfbc95fe9c77524dc3 (127.1.32.65:34893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c3efc60a6296442994fce830f0616eaa: Network error: Client connection negotiation failed: client connection to 127.1.32.65:34893: connect: Connection refused (error 111)
I20260620 23:12:11.912570 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21119
I20260620 23:12:11.917372 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 20766
2026-06-20T23:12:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (716 ms)
[----------] 6 tests from EncryptionPolicyTest (8280 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:11 dist-test-slave-fzjm krb5kdc[21290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:11 dist-test-slave-fzjm krb5kdc[21290](info): set up 2 sockets
Jun 20 23:12:11 dist-test-slave-fzjm krb5kdc[21290](info): commencing operation
krb5kdc: starting...
W20260620 23:12:12.934563 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:13 dist-test-slave-fzjm krb5kdc[21290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:13.045614 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:13.123672 21303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:13.123857 21303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:13.123878 21303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:13.124948 21303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:13.124987 21303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:13.125000 21303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:13.125013 21303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:13.125025 21303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:13.126396 21303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36847
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:13.126624 21303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:13.126833 21303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:13.131527 21308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:13.131537 21311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:13.131765 21303 server_base.cc:1034] running on GCE node
W20260620 23:12:13.131553 21309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:13.132050 21303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:13.132364 21303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:13.133530 21303 hybrid_clock.cc:648] HybridClock initialized: now 1781997133133508 us; error 32 us; skew 500 ppm
Jun 20 23:12:13 dist-test-slave-fzjm krb5kdc[21290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997133, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:13.135988 21303 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:13.136447 21303 webserver.cc:460] Webserver started at http://127.0.0.1:44811/ using document root <none> and password file <none>
I20260620 23:12:13.136669 21303 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:13.136737 21303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:13.137029 21303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:13.138145 21303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d"
format_stamp: "Formatted at 2026-06-20 23:12:13 on dist-test-slave-fzjm"
server_key: "e413713bb0167bf1900509db144458c2"
server_key_iv: "3c21cf39503efddd5909abbf7557a93c"
server_key_version: "encryptionkey@0"
I20260620 23:12:13.138509 21303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d"
format_stamp: "Formatted at 2026-06-20 23:12:13 on dist-test-slave-fzjm"
server_key: "e413713bb0167bf1900509db144458c2"
server_key_iv: "3c21cf39503efddd5909abbf7557a93c"
server_key_version: "encryptionkey@0"
I20260620 23:12:13.139916 21303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:13.140625 21318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:13.140786 21303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:13.140906 21303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d"
format_stamp: "Formatted at 2026-06-20 23:12:13 on dist-test-slave-fzjm"
server_key: "e413713bb0167bf1900509db144458c2"
server_key_iv: "3c21cf39503efddd5909abbf7557a93c"
server_key_version: "encryptionkey@0"
I20260620 23:12:13.140980 21303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:13.154838 21303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:13.156872 21303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:13.157045 21303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:13.160854 21303 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36847
I20260620 23:12:13.160887 21370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36847 every 8 connection(s)
I20260620 23:12:13.161229 21303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:13.162194 21371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:13.164397 21371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d: Bootstrap starting.
I20260620 23:12:13.164944 21371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:13.165213 21371 log.cc:824] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:13.165848 21371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d: No bootstrap required, opened a new log
I20260620 23:12:13.167469 21371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
I20260620 23:12:13.167598 21371 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:13.167620 21371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df6ecdf9cf1446b9cb0d0c1adecce7d, State: Initialized, Role: FOLLOWER
I20260620 23:12:13.167783 21371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
I20260620 23:12:13.167842 21371 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:13.167865 21371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:13.167914 21371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:13.168390 21371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
I20260620 23:12:13.168493 21371 leader_election.cc:302] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3df6ecdf9cf1446b9cb0d0c1adecce7d; no voters:
I20260620 23:12:13.168663 21371 leader_election.cc:288] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:13.168769 21374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:13.168888 21371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:13.168931 21374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [term 1 LEADER]: Becoming Leader. State: Replica: 3df6ecdf9cf1446b9cb0d0c1adecce7d, State: Running, Role: LEADER
I20260620 23:12:13.169109 21374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } }
I20260620 23:12:13.169685 21376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3df6ecdf9cf1446b9cb0d0c1adecce7d. Latest consensus state: current_term: 1 leader_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } } }
I20260620 23:12:13.169785 21376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:13.169981 21375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df6ecdf9cf1446b9cb0d0c1adecce7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36847 } } }
I20260620 23:12:13.170066 21375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:13.170157 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21303
I20260620 23:12:13.170255 21383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:13.170293 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:13.170533 1153 external_mini_cluster.cc:1428] Setting key ce395b119a3c51dbba2f23f13e6e72e8
I20260620 23:12:13.171160 21383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:13.172798 21383 catalog_manager.cc:1347] Generated new cluster ID: eef421a3ced5477f9fcae0be7e0b89bb
I20260620 23:12:13.172849 21383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:12:13 dist-test-slave-fzjm krb5kdc[21290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:12:13.179598 21383 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:13.180087 21383 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:13.182003 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:13.171036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41023 (local address 127.0.0.1:36847)
0620 23:12:13.171204 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:13.171213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:13.171251 (+ 38us) server_negotiation.cc:407] Connection header received
0620 23:12:13.171425 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:13.171432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:13.171487 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:13.171635 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:13.172419 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:13.173710 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:13.174833 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:13.175486 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:13.175599 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:13.178340 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:13.178343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:13.178344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:13.178355 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:13.180415 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:13.180826 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:13.180829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:13.180829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:13.180942 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:13.181164 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:13.181164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:13.181165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:13.181432 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:13.181585 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:13.181688 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 23:12:13.181783 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260620 23:12:13.183033 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:13.185631 21383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3df6ecdf9cf1446b9cb0d0c1adecce7d: Generated new TSK 0
I20260620 23:12:13.185796 21383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:13.192101 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:13.185531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60515 (local address 127.0.0.1:36847)
0620 23:12:13.185561 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:13.185563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:13.185943 (+ 380us) server_negotiation.cc:407] Connection header received
0620 23:12:13.185966 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:13.185966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:13.185975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:13.186037 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:13.186467 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:13.187307 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:13.187986 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:13.188269 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:13.188339 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:13.189392 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:13.189393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:13.189393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:13.189400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:13.191071 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:13.191395 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:13.191396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:13.191397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:13.191469 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:13.191671 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:13.191671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:13.191672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:13.191784 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:13.191883 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:13.191963 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:12:13.192020 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 23:12:13.193887 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21303
W20260620 23:12:13.198163 21288 connection.cc:535] client connection to 127.0.0.1:36847 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36847): Connection reset by peer (error 104)
2026-06-20T23:12:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:13 dist-test-slave-fzjm krb5kdc[21402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:13 dist-test-slave-fzjm krb5kdc[21402](info): set up 2 sockets
Jun 20 23:12:13 dist-test-slave-fzjm krb5kdc[21402](info): commencing operation
krb5kdc: starting...
W20260620 23:12:14.202687 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:14 dist-test-slave-fzjm krb5kdc[21402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:14.311794 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:14.389314 21415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:14.389496 21415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:14.389518 21415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:14.390578 21415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:14.390609 21415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:14.390623 21415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:14.390635 21415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:14.390646 21415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:14.391973 21415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:14.392141 21415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:14.392287 21415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:14.396664 21420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:14.396656 21421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:14.396720 21415 server_base.cc:1034] running on GCE node
W20260620 23:12:14.396638 21423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:14.397042 21415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:14.397285 21415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:14.398417 21415 hybrid_clock.cc:648] HybridClock initialized: now 1781997134398407 us; error 25 us; skew 500 ppm
Jun 20 23:12:14 dist-test-slave-fzjm krb5kdc[21402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:14.403733 21415 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:14.404186 21415 webserver.cc:460] Webserver started at http://127.0.0.1:34829/ using document root <none> and password file <none>
I20260620 23:12:14.404417 21415 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:14.404482 21415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:14.404626 21415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:14.405682 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "93b88618a2ae47c48a9b1ba444e20db3"
format_stamp: "Formatted at 2026-06-20 23:12:14 on dist-test-slave-fzjm"
server_key: "c3293ab5cb8aa03e2e4c2deabc0756ee"
server_key_iv: "77fe1ae097c28d11054c59d181a5765c"
server_key_version: "encryptionkey@0"
I20260620 23:12:14.406030 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "93b88618a2ae47c48a9b1ba444e20db3"
format_stamp: "Formatted at 2026-06-20 23:12:14 on dist-test-slave-fzjm"
server_key: "c3293ab5cb8aa03e2e4c2deabc0756ee"
server_key_iv: "77fe1ae097c28d11054c59d181a5765c"
server_key_version: "encryptionkey@0"
I20260620 23:12:14.407331 21415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:14.408030 21430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:14.408221 21415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:14.408303 21415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "93b88618a2ae47c48a9b1ba444e20db3"
format_stamp: "Formatted at 2026-06-20 23:12:14 on dist-test-slave-fzjm"
server_key: "c3293ab5cb8aa03e2e4c2deabc0756ee"
server_key_iv: "77fe1ae097c28d11054c59d181a5765c"
server_key_version: "encryptionkey@0"
I20260620 23:12:14.408375 21415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:14.416102 21415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:14.417733 21415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:14.417887 21415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:14.421653 21415 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41951
I20260620 23:12:14.421707 21482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41951 every 8 connection(s)
I20260620 23:12:14.422045 21415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:14.423090 21483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:14.425132 21483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3: Bootstrap starting.
I20260620 23:12:14.425698 21483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:14.425755 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21415
I20260620 23:12:14.425868 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:14.426043 21483 log.cc:824] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:14.426074 1153 external_mini_cluster.cc:1428] Setting key e903109fe1a08a14046607c0962d7cc4
I20260620 23:12:14.426982 21483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3: No bootstrap required, opened a new log
I20260620 23:12:14.429320 21483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b88618a2ae47c48a9b1ba444e20db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260620 23:12:14.429430 21483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:14.429450 21483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93b88618a2ae47c48a9b1ba444e20db3, State: Initialized, Role: FOLLOWER
I20260620 23:12:14.429564 21483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b88618a2ae47c48a9b1ba444e20db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260620 23:12:14.429626 21483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:14.429654 21483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:14.429698 21483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:14.430303 21483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b88618a2ae47c48a9b1ba444e20db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260620 23:12:14.430398 21483 leader_election.cc:302] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93b88618a2ae47c48a9b1ba444e20db3; no voters:
I20260620 23:12:14.430558 21483 leader_election.cc:288] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:14.430619 21488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:14.430781 21488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [term 1 LEADER]: Becoming Leader. State: Replica: 93b88618a2ae47c48a9b1ba444e20db3, State: Running, Role: LEADER
I20260620 23:12:14.430797 21483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:14.430868 21488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b88618a2ae47c48a9b1ba444e20db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260620 23:12:14.431253 21488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93b88618a2ae47c48a9b1ba444e20db3. Latest consensus state: current_term: 1 leader_uuid: "93b88618a2ae47c48a9b1ba444e20db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b88618a2ae47c48a9b1ba444e20db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } } }
I20260620 23:12:14.431461 21488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:14.431264 21489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93b88618a2ae47c48a9b1ba444e20db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b88618a2ae47c48a9b1ba444e20db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } } }
I20260620 23:12:14.431684 21489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:14.432015 21497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:14.432359 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:14.426535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57235 (local address 127.0.0.1:41951)
0620 23:12:14.426687 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:14.426696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:14.426726 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:12:14.427285 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:14.427300 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:14.427357 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:14.427491 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:14.427655 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:14.429030 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:14.429926 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:14.430502 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:14.430621 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:14.431503 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:14.431508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:14.431509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:14.431690 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:14.431733 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:14.431749 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:14.431841 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:14.432118 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 23:12:14.432145 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260620 23:12:14.432528 21497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:14.433277 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:14.434132 21497 catalog_manager.cc:1347] Generated new cluster ID: 24743f5d979f4c4a8f4bca7392359d27
I20260620 23:12:14.434177 21497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:14.437929 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:14.434774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33339 (local address 127.0.0.1:41951)
0620 23:12:14.434804 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:14.434806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:14.434879 (+ 73us) server_negotiation.cc:407] Connection header received
0620 23:12:14.434983 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:14.434984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:14.434993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:14.435059 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:14.435157 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:14.435837 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:14.436759 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:14.437073 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:14.437146 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:14.437705 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:14.437706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:14.437707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:14.437710 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:14.437722 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:14.437726 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:14.437786 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:14.437851 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 23:12:14.437860 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:12:14.445108 21497 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:14.445526 21497 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:14.451145 21497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93b88618a2ae47c48a9b1ba444e20db3: Generated new TSK 0
I20260620 23:12:14.451325 21497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:14.456315 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21415
W20260620 23:12:14.460460 21400 connection.cc:535] client connection to 127.0.0.1:41951 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41951): Connection reset by peer (error 104)
2026-06-20T23:12:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1262 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:14 dist-test-slave-fzjm krb5kdc[21514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:14 dist-test-slave-fzjm krb5kdc[21514](info): set up 2 sockets
Jun 20 23:12:14 dist-test-slave-fzjm krb5kdc[21514](info): commencing operation
krb5kdc: starting...
W20260620 23:12:15.466878 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:15 dist-test-slave-fzjm krb5kdc[21514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:15.575399 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33337
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:15.651502 21527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:15.651715 21527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:15.651752 21527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:15.652866 21527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:15.652930 21527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:15.652954 21527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:15.652976 21527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:15.653005 21527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:15.654403 21527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33337
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:15.654656 21527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:15.654920 21527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:15.659438 21532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:15.659444 21533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:15.659444 21535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:15.659650 21527 server_base.cc:1034] running on GCE node
I20260620 23:12:15.659914 21527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:15.660192 21527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:15.661351 21527 hybrid_clock.cc:648] HybridClock initialized: now 1781997135661354 us; error 39 us; skew 500 ppm
Jun 20 23:12:15 dist-test-slave-fzjm krb5kdc[21514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997135, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:15.663738 21527 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:15.664166 21527 webserver.cc:460] Webserver started at http://127.0.0.1:46411/ using document root <none> and password file <none>
I20260620 23:12:15.664366 21527 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:15.664404 21527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:15.664491 21527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:15.665406 21527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "eb0fde699597443d8556004c74cbb6c5"
format_stamp: "Formatted at 2026-06-20 23:12:15 on dist-test-slave-fzjm"
server_key: "80271ba47458783c468cc53b9211097c"
server_key_iv: "29a5273f2517c3fe338759106b2831ae"
server_key_version: "encryptionkey@0"
I20260620 23:12:15.665676 21527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "eb0fde699597443d8556004c74cbb6c5"
format_stamp: "Formatted at 2026-06-20 23:12:15 on dist-test-slave-fzjm"
server_key: "80271ba47458783c468cc53b9211097c"
server_key_iv: "29a5273f2517c3fe338759106b2831ae"
server_key_version: "encryptionkey@0"
I20260620 23:12:15.666849 21527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 23:12:15.667582 21542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:15.667729 21527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:15.667814 21527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "eb0fde699597443d8556004c74cbb6c5"
format_stamp: "Formatted at 2026-06-20 23:12:15 on dist-test-slave-fzjm"
server_key: "80271ba47458783c468cc53b9211097c"
server_key_iv: "29a5273f2517c3fe338759106b2831ae"
server_key_version: "encryptionkey@0"
I20260620 23:12:15.667866 21527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:15.670854 21527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:15.672739 21527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:15.672853 21527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:15.676523 21527 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33337
I20260620 23:12:15.676568 21594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33337 every 8 connection(s)
I20260620 23:12:15.676855 21527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:15.677760 21595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:15.679352 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21527
I20260620 23:12:15.679497 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:15.679723 1153 external_mini_cluster.cc:1428] Setting key aa0d318e5e7252166ca6ef11b83b2356
I20260620 23:12:15.680297 21595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5: Bootstrap starting.
I20260620 23:12:15.680904 21595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:15.681223 21595 log.cc:824] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:15.682035 21595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5: No bootstrap required, opened a new log
I20260620 23:12:15.682516 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:15.680296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38707 (local address 127.0.0.1:33337)
0620 23:12:15.680511 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:15.680518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:15.680546 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:12:15.680982 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:15.681012 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:15.681045 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:15.681182 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:15.681969 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:15.681972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:15.681972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:15.681976 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:15.682017 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:15.682027 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:15.682098 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:15.682212 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 23:12:15.682274 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260620 23:12:15.683578 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:15.684461 21595 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0fde699597443d8556004c74cbb6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33337 } }
I20260620 23:12:15.684600 21595 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:15.684688 21595 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0fde699597443d8556004c74cbb6c5, State: Initialized, Role: FOLLOWER
I20260620 23:12:15.684823 21595 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0fde699597443d8556004c74cbb6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33337 } }
I20260620 23:12:15.684937 21595 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:15.685012 21595 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:15.685070 21595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:15.685849 21595 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0fde699597443d8556004c74cbb6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33337 } }
I20260620 23:12:15.685973 21595 leader_election.cc:302] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb0fde699597443d8556004c74cbb6c5; no voters:
I20260620 23:12:15.686062 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:15.684964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33711 (local address 127.0.0.1:33337)
0620 23:12:15.685000 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:15.685002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:15.685160 (+ 158us) server_negotiation.cc:407] Connection header received
0620 23:12:15.685230 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:15.685231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:15.685237 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:15.685304 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:15.685834 (+ 530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:15.685835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:15.685835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:15.685840 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:15.685855 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:15.685860 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:15.685927 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:15.685991 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 23:12:15.686004 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:15.686151 21595 leader_election.cc:288] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:15.686326 21605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:15.686412 21595 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:15.686484 21605 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [term 1 LEADER]: Becoming Leader. State: Replica: eb0fde699597443d8556004c74cbb6c5, State: Running, Role: LEADER
I20260620 23:12:15.686556 21605 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0fde699597443d8556004c74cbb6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33337 } }
I20260620 23:12:15.686874 21605 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0fde699597443d8556004c74cbb6c5. Latest consensus state: current_term: 1 leader_uuid: "eb0fde699597443d8556004c74cbb6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0fde699597443d8556004c74cbb6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33337 } } }
I20260620 23:12:15.686940 21606 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb0fde699597443d8556004c74cbb6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0fde699597443d8556004c74cbb6c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33337 } } }
I20260620 23:12:15.686975 21605 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:15.687179 21606 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:15.687305 21613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:15.687803 21613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:15.689175 21613 catalog_manager.cc:1347] Generated new cluster ID: a0f7322b3ce14b23bb1d97554dda40ba
I20260620 23:12:15.689247 21613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:15.711588 21613 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:15.711838 21613 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:15.714294 21613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb0fde699597443d8556004c74cbb6c5: Generated new TSK 0
I20260620 23:12:15.714494 21613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:15.724912 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21527
2026-06-20T23:12:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:15 dist-test-slave-fzjm krb5kdc[21626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:15 dist-test-slave-fzjm krb5kdc[21626](info): set up 2 sockets
Jun 20 23:12:15 dist-test-slave-fzjm krb5kdc[21626](info): commencing operation
krb5kdc: starting...
W20260620 23:12:16.730710 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:16 dist-test-slave-fzjm krb5kdc[21626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:16.839749 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:16.917434 21639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:16.917703 21639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:16.917749 21639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:16.918850 21639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:16.918902 21639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:16.918954 21639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:16.918991 21639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:16.919018 21639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:16.920461 21639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:16.920706 21639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:16.920926 21639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:16.925545 21644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:16.925658 21639 server_base.cc:1034] running on GCE node
W20260620 23:12:16.925545 21647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:16.925594 21645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:16.926067 21639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:16.926390 21639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:16.927546 21639 hybrid_clock.cc:648] HybridClock initialized: now 1781997136927523 us; error 35 us; skew 500 ppm
Jun 20 23:12:16 dist-test-slave-fzjm krb5kdc[21626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:16.930140 21639 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:16.930490 21639 webserver.cc:460] Webserver started at http://127.0.0.1:33901/ using document root <none> and password file <none>
I20260620 23:12:16.930665 21639 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:16.930701 21639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:16.930783 21639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:16.931763 21639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "ac8912737b1b41c2947d9c3475d3591d"
format_stamp: "Formatted at 2026-06-20 23:12:16 on dist-test-slave-fzjm"
server_key: "f0685e67d5304867bcaba577da519da4"
server_key_iv: "c2abea60b25417599e981fd889e10e94"
server_key_version: "encryptionkey@0"
I20260620 23:12:16.932070 21639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "ac8912737b1b41c2947d9c3475d3591d"
format_stamp: "Formatted at 2026-06-20 23:12:16 on dist-test-slave-fzjm"
server_key: "f0685e67d5304867bcaba577da519da4"
server_key_iv: "c2abea60b25417599e981fd889e10e94"
server_key_version: "encryptionkey@0"
I20260620 23:12:16.933241 21639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:16.933864 21654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:16.934015 21639 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:16.934108 21639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "ac8912737b1b41c2947d9c3475d3591d"
format_stamp: "Formatted at 2026-06-20 23:12:16 on dist-test-slave-fzjm"
server_key: "f0685e67d5304867bcaba577da519da4"
server_key_iv: "c2abea60b25417599e981fd889e10e94"
server_key_version: "encryptionkey@0"
I20260620 23:12:16.934162 21639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:16.950791 21639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:16.953020 21639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:16.953126 21639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:16.956688 21639 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41599
I20260620 23:12:16.956735 21706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41599 every 8 connection(s)
I20260620 23:12:16.957041 21639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:16.957968 21707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:16.960026 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d: Bootstrap starting.
I20260620 23:12:16.960681 21707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:16.961040 21707 log.cc:824] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:16.961719 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d: No bootstrap required, opened a new log
I20260620 23:12:16.963407 21707 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8912737b1b41c2947d9c3475d3591d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260620 23:12:16.963557 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21639
I20260620 23:12:16.963583 21707 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:16.963621 21707 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac8912737b1b41c2947d9c3475d3591d, State: Initialized, Role: FOLLOWER
I20260620 23:12:16.963677 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:16.963800 21707 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8912737b1b41c2947d9c3475d3591d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260620 23:12:16.963883 21707 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:16.963894 1153 external_mini_cluster.cc:1428] Setting key da42744dff1a624d96818f5df07bb78e
I20260620 23:12:16.963912 21707 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:16.963965 21707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:16.964803 21707 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8912737b1b41c2947d9c3475d3591d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260620 23:12:16.964926 21707 leader_election.cc:302] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac8912737b1b41c2947d9c3475d3591d; no voters:
I20260620 23:12:16.965132 21707 leader_election.cc:288] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:16.965152 21712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:16.965256 21712 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [term 1 LEADER]: Becoming Leader. State: Replica: ac8912737b1b41c2947d9c3475d3591d, State: Running, Role: LEADER
I20260620 23:12:16.965381 21707 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:16.965385 21712 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8912737b1b41c2947d9c3475d3591d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260620 23:12:16.965793 21712 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac8912737b1b41c2947d9c3475d3591d. Latest consensus state: current_term: 1 leader_uuid: "ac8912737b1b41c2947d9c3475d3591d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8912737b1b41c2947d9c3475d3591d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } } }
I20260620 23:12:16.966048 21712 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:16.965955 21714 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac8912737b1b41c2947d9c3475d3591d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac8912737b1b41c2947d9c3475d3591d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } } }
I20260620 23:12:16.966205 21714 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:16.966570 21721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:16.967166 21721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:16.968627 21721 catalog_manager.cc:1347] Generated new cluster ID: e43f50b658034e428d7c88c2a6eefd59
I20260620 23:12:16.968719 21721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:12:16 dist-test-slave-fzjm krb5kdc[21626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:12:16.975126 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:16.964333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56645 (local address 127.0.0.1:41599)
0620 23:12:16.964520 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:16.964529 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:16.964558 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:12:16.964653 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:16.964658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:16.964697 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:16.964838 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:16.965636 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:16.967066 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:16.968010 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:16.968579 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:16.968695 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:16.971474 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:16.971478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:16.971478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:16.971493 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:16.973527 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:16.973878 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:16.973881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:16.973882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:16.973998 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:16.974180 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:16.974181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:16.974181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:16.974554 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:16.974716 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:16.974783 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 23:12:16.974938 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260620 23:12:16.976186 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:16.984086 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:16.977504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49103 (local address 127.0.0.1:41599)
0620 23:12:16.977536 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:16.977538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:16.977586 (+ 48us) server_negotiation.cc:407] Connection header received
0620 23:12:16.977636 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:16.977637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:16.977646 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:16.977703 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:16.978263 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:16.979008 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:16.979897 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:16.980191 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:16.980259 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:16.981413 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:16.981414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:16.981415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:16.981421 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:16.982964 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:16.983321 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:16.983322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:16.983323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:16.983377 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:16.983594 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:16.983595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:16.983595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:16.983714 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:16.983806 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:16.983932 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 23:12:16.984001 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:12:16.985374 21721 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:16.986024 21721 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:16.989107 21721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac8912737b1b41c2947d9c3475d3591d: Generated new TSK 0
I20260620 23:12:16.989264 21721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:16.991524 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21639
2026-06-20T23:12:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1266 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:17 dist-test-slave-fzjm krb5kdc[21738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:17 dist-test-slave-fzjm krb5kdc[21738](info): set up 2 sockets
Jun 20 23:12:17 dist-test-slave-fzjm krb5kdc[21738](info): commencing operation
krb5kdc: starting...
W20260620 23:12:17.990965 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:18 dist-test-slave-fzjm krb5kdc[21738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:18.097579 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:18.175839 21751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:18.176050 21751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:18.176084 21751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:18.177182 21751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:18.177256 21751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:18.177279 21751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:18.177311 21751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:18.177340 21751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:18.178732 21751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:18.179033 21751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:18.179250 21751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:18.183745 21759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:18.183728 21756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:18.183727 21757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:18.184603 21751 server_base.cc:1034] running on GCE node
I20260620 23:12:18.184828 21751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:18.185102 21751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:18.186262 21751 hybrid_clock.cc:648] HybridClock initialized: now 1781997138186242 us; error 38 us; skew 500 ppm
Jun 20 23:12:18 dist-test-slave-fzjm krb5kdc[21738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997138, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:18.188555 21751 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:18.188987 21751 webserver.cc:460] Webserver started at http://127.0.0.1:38567/ using document root <none> and password file <none>
I20260620 23:12:18.189205 21751 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:18.189284 21751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:18.189425 21751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:18.190367 21751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "c830c22960b048d6a46d6f479eb990e5"
format_stamp: "Formatted at 2026-06-20 23:12:18 on dist-test-slave-fzjm"
server_key: "0c6b00a317f23fbc758407420aed262a"
server_key_iv: "6182050b81dead0d95b1474053b64076"
server_key_version: "encryptionkey@0"
I20260620 23:12:18.190706 21751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "c830c22960b048d6a46d6f479eb990e5"
format_stamp: "Formatted at 2026-06-20 23:12:18 on dist-test-slave-fzjm"
server_key: "0c6b00a317f23fbc758407420aed262a"
server_key_iv: "6182050b81dead0d95b1474053b64076"
server_key_version: "encryptionkey@0"
I20260620 23:12:18.191978 21751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:18.192688 21766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:18.192868 21751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:18.192960 21751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "c830c22960b048d6a46d6f479eb990e5"
format_stamp: "Formatted at 2026-06-20 23:12:18 on dist-test-slave-fzjm"
server_key: "0c6b00a317f23fbc758407420aed262a"
server_key_iv: "6182050b81dead0d95b1474053b64076"
server_key_version: "encryptionkey@0"
I20260620 23:12:18.193039 21751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:18.222656 21751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:18.224483 21751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:18.224634 21751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:18.228956 21751 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36093
I20260620 23:12:18.229000 21818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36093 every 8 connection(s)
I20260620 23:12:18.229349 21751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:18.230242 21819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:18.231941 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21751
I20260620 23:12:18.232074 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:18.232255 1153 external_mini_cluster.cc:1428] Setting key 26412a893dd815965fae2d6820c70c00
I20260620 23:12:18.232848 21819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5: Bootstrap starting.
I20260620 23:12:18.233433 21819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:18.233754 21819 log.cc:824] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:18.234417 21819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5: No bootstrap required, opened a new log
I20260620 23:12:18.236166 21819 raft_consensus.cc:348] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c830c22960b048d6a46d6f479eb990e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36093 } }
I20260620 23:12:18.236328 21819 raft_consensus.cc:374] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:18.236374 21819 raft_consensus.cc:729] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c830c22960b048d6a46d6f479eb990e5, State: Initialized, Role: FOLLOWER
I20260620 23:12:18.236538 21819 consensus_queue.cc:260] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c830c22960b048d6a46d6f479eb990e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36093 } }
I20260620 23:12:18.236622 21819 raft_consensus.cc:388] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:18.236661 21819 raft_consensus.cc:482] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:18.236707 21819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:18.237218 21819 raft_consensus.cc:504] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c830c22960b048d6a46d6f479eb990e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36093 } }
I20260620 23:12:18.237341 21819 leader_election.cc:302] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c830c22960b048d6a46d6f479eb990e5; no voters:
I20260620 23:12:18.237515 21819 leader_election.cc:288] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:18.237540 21824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:18.237650 21824 raft_consensus.cc:686] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [term 1 LEADER]: Becoming Leader. State: Replica: c830c22960b048d6a46d6f479eb990e5, State: Running, Role: LEADER
I20260620 23:12:18.237792 21819 sys_catalog.cc:564] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:18.237802 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:18.232810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33869 (local address 127.0.0.1:36093)
0620 23:12:18.233007 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:18.233014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:18.233037 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:12:18.233471 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:18.233478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:18.233573 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:18.233766 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:18.233969 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:18.234930 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:18.235689 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:18.236089 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:18.236209 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:18.236983 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:18.236987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:18.236987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:18.237166 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:18.237196 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:18.237207 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:18.237289 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:18.237496 (+ 207us) server_negotiation.cc:299] Negotiation successful
0620 23:12:18.237566 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260620 23:12:18.237823 21824 consensus_queue.cc:237] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c830c22960b048d6a46d6f479eb990e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36093 } }
I20260620 23:12:18.238482 21824 sys_catalog.cc:455] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c830c22960b048d6a46d6f479eb990e5. Latest consensus state: current_term: 1 leader_uuid: "c830c22960b048d6a46d6f479eb990e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c830c22960b048d6a46d6f479eb990e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36093 } } }
I20260620 23:12:18.238859 21824 sys_catalog.cc:458] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:18.238968 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:18.238782 21825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c830c22960b048d6a46d6f479eb990e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c830c22960b048d6a46d6f479eb990e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36093 } } }
I20260620 23:12:18.239086 21825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5 [sys.catalog]: This master's current role is: LEADER
W20260620 23:12:18.239569 21839 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:18.239643 21839 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:18.239715 21838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:18.239929 21838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:18.243362 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:18.240528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51599 (local address 127.0.0.1:36093)
0620 23:12:18.240567 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:18.240570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:18.240718 (+ 148us) server_negotiation.cc:407] Connection header received
0620 23:12:18.240796 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:18.240797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:18.240806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:18.240861 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:18.240982 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:18.241443 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:18.242127 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:18.242337 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:18.242437 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:18.243134 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:18.243136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:18.243136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:18.243140 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:18.243155 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:18.243158 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:18.243211 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:18.243264 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 23:12:18.243284 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:18.247967 21838 catalog_manager.cc:1347] Generated new cluster ID: a37056daeff34097b42ad9f3778b8e08
I20260620 23:12:18.248019 21838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:18.260943 21838 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:18.261480 21838 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:18.263849 21838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c830c22960b048d6a46d6f479eb990e5: Generated new TSK 0
I20260620 23:12:18.264008 21838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:18.271092 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21751
2026-06-20T23:12:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:18 dist-test-slave-fzjm krb5kdc[21850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:18 dist-test-slave-fzjm krb5kdc[21850](info): set up 2 sockets
Jun 20 23:12:18 dist-test-slave-fzjm krb5kdc[21850](info): commencing operation
krb5kdc: starting...
W20260620 23:12:19.274861 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:19 dist-test-slave-fzjm krb5kdc[21850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:19.382601 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:19.459502 21863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:19.459715 21863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:19.459751 21863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:19.460879 21863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:19.460947 21863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:19.460971 21863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:19.460999 21863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:19.461030 21863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:19.462419 21863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:19.462688 21863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:19.462918 21863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:12:19.467639 21863 server_base.cc:1034] running on GCE node
W20260620 23:12:19.467604 21869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:19.467644 21871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:19.467615 21868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:19.468173 21863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:19.468458 21863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:19.469620 21863 hybrid_clock.cc:648] HybridClock initialized: now 1781997139469600 us; error 32 us; skew 500 ppm
Jun 20 23:12:19 dist-test-slave-fzjm krb5kdc[21850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997139, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:19.474856 21863 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:19.475349 21863 webserver.cc:460] Webserver started at http://127.0.0.1:38295/ using document root <none> and password file <none>
I20260620 23:12:19.475562 21863 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:19.475633 21863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:19.475778 21863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:19.476773 21863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "e03857b6f3ee4144b2987bec1f4e4b8c"
format_stamp: "Formatted at 2026-06-20 23:12:19 on dist-test-slave-fzjm"
server_key: "1bd0fe97deb275eaf1bb06447b48f424"
server_key_iv: "0e06fecdec8dc26e48a09b07e1a6fdb8"
server_key_version: "encryptionkey@0"
I20260620 23:12:19.477150 21863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "e03857b6f3ee4144b2987bec1f4e4b8c"
format_stamp: "Formatted at 2026-06-20 23:12:19 on dist-test-slave-fzjm"
server_key: "1bd0fe97deb275eaf1bb06447b48f424"
server_key_iv: "0e06fecdec8dc26e48a09b07e1a6fdb8"
server_key_version: "encryptionkey@0"
I20260620 23:12:19.478432 21863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:19.479212 21878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:19.479429 21863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:12:19.479537 21863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "e03857b6f3ee4144b2987bec1f4e4b8c"
format_stamp: "Formatted at 2026-06-20 23:12:19 on dist-test-slave-fzjm"
server_key: "1bd0fe97deb275eaf1bb06447b48f424"
server_key_iv: "0e06fecdec8dc26e48a09b07e1a6fdb8"
server_key_version: "encryptionkey@0"
I20260620 23:12:19.479650 21863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:19.482846 21863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:19.485110 21863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:19.485280 21863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:19.489392 21863 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45291
I20260620 23:12:19.489427 21930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45291 every 8 connection(s)
I20260620 23:12:19.489733 21863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:19.490610 21931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:19.492880 21931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c: Bootstrap starting.
I20260620 23:12:19.493454 21931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:19.493762 21931 log.cc:824] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:19.494416 21931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c: No bootstrap required, opened a new log
I20260620 23:12:19.496058 21931 raft_consensus.cc:348] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260620 23:12:19.496193 21931 raft_consensus.cc:374] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:19.496248 21931 raft_consensus.cc:729] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e03857b6f3ee4144b2987bec1f4e4b8c, State: Initialized, Role: FOLLOWER
I20260620 23:12:19.496415 21931 consensus_queue.cc:260] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260620 23:12:19.496481 21931 raft_consensus.cc:388] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:19.496554 21931 raft_consensus.cc:482] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:19.496608 21931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:19.496730 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21863
I20260620 23:12:19.496876 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:19.497059 1153 external_mini_cluster.cc:1428] Setting key 31fad4bdf4985fc0db912c6e5162de0e
I20260620 23:12:19.497280 21931 raft_consensus.cc:504] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260620 23:12:19.497400 21931 leader_election.cc:302] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e03857b6f3ee4144b2987bec1f4e4b8c; no voters:
I20260620 23:12:19.497634 21931 leader_election.cc:288] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:19.497853 21937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:19.497915 21931 sys_catalog.cc:564] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:19.498037 21937 raft_consensus.cc:686] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [term 1 LEADER]: Becoming Leader. State: Replica: e03857b6f3ee4144b2987bec1f4e4b8c, State: Running, Role: LEADER
I20260620 23:12:19.498183 21937 consensus_queue.cc:237] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } }
I20260620 23:12:19.498534 21937 sys_catalog.cc:455] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } } }
I20260620 23:12:19.498994 21937 sys_catalog.cc:458] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:19.499312 21937 sys_catalog.cc:455] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e03857b6f3ee4144b2987bec1f4e4b8c. Latest consensus state: current_term: 1 leader_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e03857b6f3ee4144b2987bec1f4e4b8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45291 } } }
I20260620 23:12:19.499449 21937 sys_catalog.cc:458] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:19.499706 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:19.497506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51795 (local address 127.0.0.1:45291)
0620 23:12:19.497683 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:19.497691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:19.497725 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:12:19.497892 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:19.497900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:19.497933 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:19.498087 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:19.499089 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:19.499093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:19.499093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:19.499098 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:19.499148 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:19.499164 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:19.499227 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:19.499355 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 23:12:19.499396 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260620 23:12:19.499917 21944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:19.500487 21944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:19.500618 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:19.502058 21944 catalog_manager.cc:1347] Generated new cluster ID: cdbfca09484b46e288186fe380bb3805
I20260620 23:12:19.502247 21944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:19.503327 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:19.502078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41979 (local address 127.0.0.1:45291)
0620 23:12:19.502129 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:19.502131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:19.502425 (+ 294us) server_negotiation.cc:407] Connection header received
0620 23:12:19.502460 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:19.502461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:19.502466 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:19.502553 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:19.503108 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:19.503109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:19.503110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:19.503114 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:19.503129 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:19.503135 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:19.503206 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:19.503261 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 23:12:19.503273 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 23:12:19.509805 21944 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:19.509943 21944 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:19.523012 21944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e03857b6f3ee4144b2987bec1f4e4b8c: Generated new TSK 0
I20260620 23:12:19.523205 21944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:19.525823 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21863
2026-06-20T23:12:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:19 dist-test-slave-fzjm krb5kdc[21962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:19 dist-test-slave-fzjm krb5kdc[21962](info): set up 2 sockets
Jun 20 23:12:19 dist-test-slave-fzjm krb5kdc[21962](info): commencing operation
krb5kdc: starting...
W20260620 23:12:20.523099 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:20 dist-test-slave-fzjm krb5kdc[21962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:20.631735 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:20.707932 21975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:20.708115 21975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:20.708137 21975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:20.709250 21975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:20.709290 21975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:20.709304 21975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:20.709317 21975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:20.709376 21975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:20.710721 21975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:20.710987 21975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:20.711202 21975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:20.715669 21980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:20.715714 21981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:20.715679 21983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:20.715915 21975 server_base.cc:1034] running on GCE node
I20260620 23:12:20.716138 21975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:20.716403 21975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:20.717554 21975 hybrid_clock.cc:648] HybridClock initialized: now 1781997140717548 us; error 33 us; skew 500 ppm
Jun 20 23:12:20 dist-test-slave-fzjm krb5kdc[21962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997140, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:20.720005 21975 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:20.720419 21975 webserver.cc:460] Webserver started at http://127.0.0.1:41799/ using document root <none> and password file <none>
I20260620 23:12:20.720635 21975 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:20.720705 21975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:20.720858 21975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:20.721861 21975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068"
format_stamp: "Formatted at 2026-06-20 23:12:20 on dist-test-slave-fzjm"
server_key: "6c4c63649b05d97c35d2270ba9698b24"
server_key_iv: "c54cc86614dbc45f37713c47b318ccab"
server_key_version: "encryptionkey@0"
I20260620 23:12:20.722220 21975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068"
format_stamp: "Formatted at 2026-06-20 23:12:20 on dist-test-slave-fzjm"
server_key: "6c4c63649b05d97c35d2270ba9698b24"
server_key_iv: "c54cc86614dbc45f37713c47b318ccab"
server_key_version: "encryptionkey@0"
I20260620 23:12:20.723613 21975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:20.724334 21990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:20.724496 21975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:20.724602 21975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068"
format_stamp: "Formatted at 2026-06-20 23:12:20 on dist-test-slave-fzjm"
server_key: "6c4c63649b05d97c35d2270ba9698b24"
server_key_iv: "c54cc86614dbc45f37713c47b318ccab"
server_key_version: "encryptionkey@0"
I20260620 23:12:20.724674 21975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:20.736485 21975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:20.738806 21975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:20.738989 21975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:20.742847 21975 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38967
I20260620 23:12:20.742942 22042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38967 every 8 connection(s)
I20260620 23:12:20.743337 21975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:20.744232 22043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:20.745605 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 21975
I20260620 23:12:20.745726 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:20.745942 1153 external_mini_cluster.cc:1428] Setting key 4666494eb12ff3561ff80d218343a10e
I20260620 23:12:20.746786 22043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068: Bootstrap starting.
I20260620 23:12:20.747411 22043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:20.747745 22043 log.cc:824] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:20.748576 22043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068: No bootstrap required, opened a new log
I20260620 23:12:20.750672 22043 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260620 23:12:20.750849 22043 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:20.750911 22043 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c9b1b47c0b45eca8e6e86ef1e4c068, State: Initialized, Role: FOLLOWER
I20260620 23:12:20.751057 22043 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260620 23:12:20.751171 22043 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:20.751215 22043 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:20.751283 22043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:20.751787 22043 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260620 23:12:20.751905 22043 leader_election.cc:302] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c9b1b47c0b45eca8e6e86ef1e4c068; no voters:
I20260620 23:12:20.752100 22043 leader_election.cc:288] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:20.752125 22048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:20.752233 22048 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [term 1 LEADER]: Becoming Leader. State: Replica: b8c9b1b47c0b45eca8e6e86ef1e4c068, State: Running, Role: LEADER
I20260620 23:12:20.752314 22048 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260620 23:12:20.752357 22043 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:20.752619 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } } }
I20260620 23:12:20.752686 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:20.752802 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c9b1b47c0b45eca8e6e86ef1e4c068. Latest consensus state: current_term: 1 leader_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c9b1b47c0b45eca8e6e86ef1e4c068" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } } }
I20260620 23:12:20.752884 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068 [sys.catalog]: This master's current role is: LEADER
Jun 20 23:12:20 dist-test-slave-fzjm krb5kdc[21962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260620 23:12:20.754137 22064 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:20.754222 22064 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:20.754329 22053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:20.754503 22053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:20.756069 22053 catalog_manager.cc:1347] Generated new cluster ID: ffa53bd1ef6a4b56822fc0fa0f304869
I20260620 23:12:20.756119 22053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:20.757495 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:20.746471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:50897 (local address 127.0.0.1:38967)
0620 23:12:20.746643 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:20.746656 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:20.746681 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:12:20.747156 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:20.747163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:20.747211 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:20.747333 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:20.748100 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:20.749188 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:20.750304 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:20.750699 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:20.750790 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:20.753779 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:20.753783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:20.753783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:20.753796 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:20.755779 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:20.756243 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:20.756245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:20.756246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:20.756349 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:20.756618 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:20.756619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:20.756619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:20.756882 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:20.757014 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:20.757207 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 23:12:20.757297 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260620 23:12:20.758440 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:20.767005 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:20.759858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:57049 (local address 127.0.0.1:38967)
0620 23:12:20.759892 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:20.759893 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:20.760175 (+ 282us) server_negotiation.cc:407] Connection header received
0620 23:12:20.760199 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:20.760200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:20.760211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:20.760267 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:20.760817 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:20.761569 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:20.762501 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:20.762834 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:20.762934 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:20.764149 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:20.764150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:20.764151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:20.764157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:20.765741 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:20.766161 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:20.766162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:20.766162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:20.766224 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:20.766453 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:20.766454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:20.766455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:20.766583 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:20.766685 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:20.766801 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 23:12:20.766864 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:12:20.771411 22053 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:20.771880 22053 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:20.782037 22053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8c9b1b47c0b45eca8e6e86ef1e4c068: Generated new TSK 0
I20260620 23:12:20.782197 22053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:20.789234 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 21975
W20260620 23:12:20.793507 21960 connection.cc:535] client connection to 127.0.0.1:38967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38967): Connection reset by peer (error 104)
2026-06-20T23:12:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:20 dist-test-slave-fzjm krb5kdc[22074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:20 dist-test-slave-fzjm krb5kdc[22074](info): set up 2 sockets
Jun 20 23:12:20 dist-test-slave-fzjm krb5kdc[22074](info): commencing operation
krb5kdc: starting...
W20260620 23:12:21.782863 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:21 dist-test-slave-fzjm krb5kdc[22074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:21.891566 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:21.968916 22087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:21.969101 22087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:21.969123 22087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:21.970247 22087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:21.970288 22087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:21.970301 22087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:21.970340 22087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:21.970369 22087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:21.971763 22087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:21.971999 22087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:21.972219 22087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:21.976677 22095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:21.976701 22092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:21.976728 22093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:21.976894 22087 server_base.cc:1034] running on GCE node
I20260620 23:12:21.977180 22087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:21.977443 22087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:21.978605 22087 hybrid_clock.cc:648] HybridClock initialized: now 1781997141978591 us; error 25 us; skew 500 ppm
Jun 20 23:12:21 dist-test-slave-fzjm krb5kdc[22074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:21.980849 22087 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:21.981287 22087 webserver.cc:460] Webserver started at http://127.0.0.1:33485/ using document root <none> and password file <none>
I20260620 23:12:21.981511 22087 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:21.981576 22087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:21.981719 22087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:21.982756 22087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "fd4272bf71394cae98cb0efd28ad9f1c"
format_stamp: "Formatted at 2026-06-20 23:12:21 on dist-test-slave-fzjm"
server_key: "dd45d52eb25fd6064c84c0f16686fc17"
server_key_iv: "73161351eae16a876023ac8c4a5e09a9"
server_key_version: "encryptionkey@0"
I20260620 23:12:21.983135 22087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "fd4272bf71394cae98cb0efd28ad9f1c"
format_stamp: "Formatted at 2026-06-20 23:12:21 on dist-test-slave-fzjm"
server_key: "dd45d52eb25fd6064c84c0f16686fc17"
server_key_iv: "73161351eae16a876023ac8c4a5e09a9"
server_key_version: "encryptionkey@0"
I20260620 23:12:21.984333 22087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:12:21.984985 22102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:21.985167 22087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:21.985278 22087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "fd4272bf71394cae98cb0efd28ad9f1c"
format_stamp: "Formatted at 2026-06-20 23:12:21 on dist-test-slave-fzjm"
server_key: "dd45d52eb25fd6064c84c0f16686fc17"
server_key_iv: "73161351eae16a876023ac8c4a5e09a9"
server_key_version: "encryptionkey@0"
I20260620 23:12:21.985348 22087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:21.995944 22087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:21.997568 22087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:21.997713 22087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:22.001585 22087 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39449
I20260620 23:12:22.001638 22154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39449 every 8 connection(s)
I20260620 23:12:22.001951 22087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:22.002919 22155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:22.005355 22155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c: Bootstrap starting.
I20260620 23:12:22.005539 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22087
I20260620 23:12:22.005661 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:22.005975 1153 external_mini_cluster.cc:1428] Setting key f76fff049875fc2c66aeeadb4cacd63d
I20260620 23:12:22.006140 22155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:22.006464 22155 log.cc:824] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:22.007197 22155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c: No bootstrap required, opened a new log
I20260620 23:12:22.009239 22155 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260620 23:12:22.009395 22155 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:22.009428 22155 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4272bf71394cae98cb0efd28ad9f1c, State: Initialized, Role: FOLLOWER
I20260620 23:12:22.009531 22155 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260620 23:12:22.009594 22155 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:22.009635 22155 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:22.009677 22155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:22.010195 22155 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260620 23:12:22.010324 22155 leader_election.cc:302] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd4272bf71394cae98cb0efd28ad9f1c; no voters:
I20260620 23:12:22.010484 22155 leader_election.cc:288] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:22.010545 22160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:22.010690 22160 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [term 1 LEADER]: Becoming Leader. State: Replica: fd4272bf71394cae98cb0efd28ad9f1c, State: Running, Role: LEADER
I20260620 23:12:22.010771 22155 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:22.010867 22160 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260620 23:12:22.011304 22162 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } } }
I20260620 23:12:22.011438 22162 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:22.011699 22169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:22.011869 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:22.006521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:59283 (local address 127.0.0.1:39449)
0620 23:12:22.006690 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:22.006700 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:22.006784 (+ 84us) server_negotiation.cc:407] Connection header received
0620 23:12:22.007363 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:22.007370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:22.007422 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:22.007549 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:22.007787 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:22.008661 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:22.009796 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:22.010179 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:22.010267 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:22.011031 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:22.011035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:22.011035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:22.011226 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:22.011286 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:22.011303 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:22.011387 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:22.011651 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 23:12:22.011678 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260620 23:12:22.012326 22169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:22.012804 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:22.012420 22161 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd4272bf71394cae98cb0efd28ad9f1c. Latest consensus state: current_term: 1 leader_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4272bf71394cae98cb0efd28ad9f1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } } }
I20260620 23:12:22.013377 22161 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:22.013962 22169 catalog_manager.cc:1347] Generated new cluster ID: d9c98e6252ea451c958ebba4b83793c4
I20260620 23:12:22.014012 22169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:22.016916 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:22.014229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:53191 (local address 127.0.0.1:39449)
0620 23:12:22.014271 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:22.014273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:22.014346 (+ 73us) server_negotiation.cc:407] Connection header received
0620 23:12:22.014413 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:22.014413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:22.014423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:22.014475 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:22.014594 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:22.015125 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:22.015810 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:22.016021 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:22.016090 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:22.016701 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:22.016703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:22.016703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:22.016707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:22.016720 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:22.016723 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:22.016782 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:22.016839 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 23:12:22.016847 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:12:22.029448 22169 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:22.029987 22169 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:22.033632 22169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd4272bf71394cae98cb0efd28ad9f1c: Generated new TSK 0
I20260620 23:12:22.033797 22169 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:22.043267 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22087
W20260620 23:12:22.047397 22072 connection.cc:535] client connection to 127.0.0.1:39449 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39449): Connection reset by peer (error 104)
2026-06-20T23:12:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:22 dist-test-slave-fzjm krb5kdc[22186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:22 dist-test-slave-fzjm krb5kdc[22186](info): set up 2 sockets
Jun 20 23:12:22 dist-test-slave-fzjm krb5kdc[22186](info): commencing operation
krb5kdc: starting...
W20260620 23:12:23.031028 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:23 dist-test-slave-fzjm krb5kdc[22186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:23.141258 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:23.220805 22199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:23.220991 22199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:23.221014 22199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:23.222074 22199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:23.222114 22199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:23.222127 22199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:23.222138 22199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:23.222149 22199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:23.223739 22199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:23.223999 22199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:23.224212 22199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:23.229112 22205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:23.229137 22204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:23.229141 22199 server_base.cc:1034] running on GCE node
W20260620 23:12:23.229104 22207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:23.229584 22199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:23.229915 22199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:23.231081 22199 hybrid_clock.cc:648] HybridClock initialized: now 1781997143231053 us; error 56 us; skew 500 ppm
Jun 20 23:12:23 dist-test-slave-fzjm krb5kdc[22186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997143, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:23.233428 22199 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:23.233902 22199 webserver.cc:460] Webserver started at http://127.0.0.1:45577/ using document root <none> and password file <none>
I20260620 23:12:23.234122 22199 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:23.234185 22199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:23.234367 22199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:23.235522 22199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd"
format_stamp: "Formatted at 2026-06-20 23:12:23 on dist-test-slave-fzjm"
server_key: "a2f131aa3cfc6341fc275e4f17c8bc41"
server_key_iv: "a3e62ffa547840a1e428ed605322d4d6"
server_key_version: "encryptionkey@0"
I20260620 23:12:23.235883 22199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd"
format_stamp: "Formatted at 2026-06-20 23:12:23 on dist-test-slave-fzjm"
server_key: "a2f131aa3cfc6341fc275e4f17c8bc41"
server_key_iv: "a3e62ffa547840a1e428ed605322d4d6"
server_key_version: "encryptionkey@0"
I20260620 23:12:23.237221 22199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:23.237937 22214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:23.238111 22199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:23.238204 22199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd"
format_stamp: "Formatted at 2026-06-20 23:12:23 on dist-test-slave-fzjm"
server_key: "a2f131aa3cfc6341fc275e4f17c8bc41"
server_key_iv: "a3e62ffa547840a1e428ed605322d4d6"
server_key_version: "encryptionkey@0"
I20260620 23:12:23.238289 22199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:23.241638 22199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:23.243610 22199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:23.243772 22199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:23.247758 22199 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40251
I20260620 23:12:23.247794 22266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40251 every 8 connection(s)
I20260620 23:12:23.248116 22199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:23.249022 22267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:23.251371 22267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd: Bootstrap starting.
I20260620 23:12:23.251936 22267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:23.252228 22267 log.cc:824] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:23.252940 22267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd: No bootstrap required, opened a new log
I20260620 23:12:23.254577 22267 raft_consensus.cc:348] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40251 } }
I20260620 23:12:23.254704 22267 raft_consensus.cc:374] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:23.254726 22267 raft_consensus.cc:729] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e16bdc30fbbd4a7baf5406cc113b9fdd, State: Initialized, Role: FOLLOWER
I20260620 23:12:23.254920 22267 consensus_queue.cc:260] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40251 } }
I20260620 23:12:23.255007 22267 raft_consensus.cc:388] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:23.255045 22267 raft_consensus.cc:482] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:23.255093 22267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:23.255483 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22199
I20260620 23:12:23.255626 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:23.255614 22267 raft_consensus.cc:504] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40251 } }
I20260620 23:12:23.255731 22267 leader_election.cc:302] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e16bdc30fbbd4a7baf5406cc113b9fdd; no voters:
I20260620 23:12:23.255862 1153 external_mini_cluster.cc:1428] Setting key 88db1b8016d6496bd60d74653de2966b
I20260620 23:12:23.255906 22267 leader_election.cc:288] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:23.255934 22270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:23.256093 22270 raft_consensus.cc:686] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [term 1 LEADER]: Becoming Leader. State: Replica: e16bdc30fbbd4a7baf5406cc113b9fdd, State: Running, Role: LEADER
I20260620 23:12:23.256148 22267 sys_catalog.cc:564] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:23.256259 22270 consensus_queue.cc:237] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40251 } }
I20260620 23:12:23.256644 22270 sys_catalog.cc:455] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e16bdc30fbbd4a7baf5406cc113b9fdd. Latest consensus state: current_term: 1 leader_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40251 } } }
I20260620 23:12:23.256657 22271 sys_catalog.cc:455] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16bdc30fbbd4a7baf5406cc113b9fdd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40251 } } }
I20260620 23:12:23.256762 22270 sys_catalog.cc:458] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:23.256778 22271 sys_catalog.cc:458] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:23.256960 22276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:23.257515 22276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:23.258739 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:23.256340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:45505 (local address 127.0.0.1:40251)
0620 23:12:23.256672 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:23.256681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:23.256738 (+ 57us) server_negotiation.cc:407] Connection header received
0620 23:12:23.256856 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:23.256863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:23.256894 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:23.257042 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:23.258136 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:23.258140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:23.258140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:23.258145 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:23.258193 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:23.258207 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:23.258267 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:23.258514 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 23:12:23.258558 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260620 23:12:23.259862 22276 catalog_manager.cc:1347] Generated new cluster ID: 443a2c720cb64af2a244cf11bb8f9cac
I20260620 23:12:23.259900 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:23.259917 22276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:23.262292 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:23.261270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:56557 (local address 127.0.0.1:40251)
0620 23:12:23.261296 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:23.261298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:23.261443 (+ 145us) server_negotiation.cc:407] Connection header received
0620 23:12:23.261503 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:23.261503 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:23.261507 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:23.261552 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:23.262098 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:23.262099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:23.262099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:23.262102 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:23.262114 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:23.262117 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:23.262158 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:23.262223 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 23:12:23.262234 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260620 23:12:23.266673 22276 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:23.266858 22276 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:23.271174 22276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e16bdc30fbbd4a7baf5406cc113b9fdd: Generated new TSK 0
I20260620 23:12:23.271319 22276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:23.273804 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22199
2026-06-20T23:12:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1230 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:23 dist-test-slave-fzjm krb5kdc[22298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:23 dist-test-slave-fzjm krb5kdc[22298](info): set up 2 sockets
Jun 20 23:12:23 dist-test-slave-fzjm krb5kdc[22298](info): commencing operation
krb5kdc: starting...
W20260620 23:12:24.279045 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:24 dist-test-slave-fzjm krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:24.391114 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:24.469081 22311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:24.469265 22311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:24.469287 22311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:24.470376 22311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:24.470412 22311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:24.470427 22311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:24.470438 22311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:24.470499 22311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:24.471879 22311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:24.472100 22311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:24.472316 22311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:24.477046 22317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:24.477056 22319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:24.477056 22316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:24.477370 22311 server_base.cc:1034] running on GCE node
I20260620 23:12:24.477614 22311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:24.477927 22311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:24.479069 22311 hybrid_clock.cc:648] HybridClock initialized: now 1781997144479046 us; error 37 us; skew 500 ppm
Jun 20 23:12:24 dist-test-slave-fzjm krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:24.486342 22311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:24.486863 22311 webserver.cc:460] Webserver started at http://127.0.0.1:44359/ using document root <none> and password file <none>
I20260620 23:12:24.487185 22311 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:24.487274 22311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:24.487432 22311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:24.488484 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "729abb7939404057abb972f30dd334b4"
format_stamp: "Formatted at 2026-06-20 23:12:24 on dist-test-slave-fzjm"
server_key: "f4d0541f0604778d444dc82d94d21eb5"
server_key_iv: "c16300cd333f403add4ebc97ae120e3a"
server_key_version: "encryptionkey@0"
I20260620 23:12:24.488838 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "729abb7939404057abb972f30dd334b4"
format_stamp: "Formatted at 2026-06-20 23:12:24 on dist-test-slave-fzjm"
server_key: "f4d0541f0604778d444dc82d94d21eb5"
server_key_iv: "c16300cd333f403add4ebc97ae120e3a"
server_key_version: "encryptionkey@0"
I20260620 23:12:24.490111 22311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:24.490882 22326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:24.491082 22311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:24.491180 22311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "729abb7939404057abb972f30dd334b4"
format_stamp: "Formatted at 2026-06-20 23:12:24 on dist-test-slave-fzjm"
server_key: "f4d0541f0604778d444dc82d94d21eb5"
server_key_iv: "c16300cd333f403add4ebc97ae120e3a"
server_key_version: "encryptionkey@0"
I20260620 23:12:24.491256 22311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:24.499953 22311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:24.501610 22311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:24.501709 22311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:24.505520 22311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41579
I20260620 23:12:24.505604 22378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41579 every 8 connection(s)
I20260620 23:12:24.505925 22311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:24.506841 22379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:24.508948 22379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4: Bootstrap starting.
I20260620 23:12:24.509421 22379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:24.509641 22379 log.cc:824] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:24.510250 22379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4: No bootstrap required, opened a new log
I20260620 23:12:24.511782 22379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729abb7939404057abb972f30dd334b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260620 23:12:24.511909 22379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:24.511929 22379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 729abb7939404057abb972f30dd334b4, State: Initialized, Role: FOLLOWER
I20260620 23:12:24.512020 22379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729abb7939404057abb972f30dd334b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260620 23:12:24.512094 22379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:24.512130 22379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:24.512198 22379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:24.512687 22379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729abb7939404057abb972f30dd334b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260620 23:12:24.512802 22379 leader_election.cc:302] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 729abb7939404057abb972f30dd334b4; no voters:
I20260620 23:12:24.512989 22379 leader_election.cc:288] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:24.513072 22382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:24.513226 22382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [term 1 LEADER]: Becoming Leader. State: Replica: 729abb7939404057abb972f30dd334b4, State: Running, Role: LEADER
I20260620 23:12:24.513242 22379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:24.513365 22382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729abb7939404057abb972f30dd334b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260620 23:12:24.513932 22384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 729abb7939404057abb972f30dd334b4. Latest consensus state: current_term: 1 leader_uuid: "729abb7939404057abb972f30dd334b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729abb7939404057abb972f30dd334b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } } }
I20260620 23:12:24.514034 22384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:24.514183 22383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "729abb7939404057abb972f30dd334b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729abb7939404057abb972f30dd334b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } } }
I20260620 23:12:24.514299 22383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:24.514365 22391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:24.514791 22391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:24.515471 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22311
I20260620 23:12:24.515630 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:24.515831 1153 external_mini_cluster.cc:1428] Setting key defa7e352c2e5da76e67e207bef8349f
I20260620 23:12:24.516655 22391 catalog_manager.cc:1347] Generated new cluster ID: 0b9bdd32fcef43b3b888f3743e7e0e88
I20260620 23:12:24.516713 22391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:12:24 dist-test-slave-fzjm krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:12:24.527491 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:24.516294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:60161 (local address 127.0.0.1:41579)
0620 23:12:24.516723 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:24.516733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:24.516764 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:12:24.516883 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:24.516889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:24.516941 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:24.517064 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:24.517807 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:24.519148 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:24.520245 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:24.520857 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:24.520965 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:24.523755 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:24.523759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:24.523760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:24.523771 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:24.525808 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:24.526266 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:24.526269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:24.526270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:24.526373 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:24.526598 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:24.526598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:24.526599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:24.526904 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:24.527062 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:24.527159 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 23:12:24.527272 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":67,"threads_started":1}
I20260620 23:12:24.528662 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:24.537068 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:24.530117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:49513 (local address 127.0.0.1:41579)
0620 23:12:24.530152 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:24.530154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:24.530178 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:12:24.530258 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:24.530259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:24.530274 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:24.530317 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:24.530919 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:24.531637 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:24.532642 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:24.532930 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:24.532983 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:24.534333 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:24.534334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:24.534335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:24.534341 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:24.535907 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:24.536332 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:24.536333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:24.536333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:24.536385 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:24.536609 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:24.536609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:24.536610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:24.536712 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:24.536805 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:24.536933 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 23:12:24.536990 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 23:12:24.539938 22391 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:24.540414 22391 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:24.546667 22391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 729abb7939404057abb972f30dd334b4: Generated new TSK 0
I20260620 23:12:24.546872 22391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:24.553268 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22311
W20260620 23:12:24.557756 22296 connection.cc:535] client connection to 127.0.0.1:41579 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41579): Connection reset by peer (error 104)
2026-06-20T23:12:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:24 dist-test-slave-fzjm krb5kdc[22410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:24 dist-test-slave-fzjm krb5kdc[22410](info): set up 2 sockets
Jun 20 23:12:24 dist-test-slave-fzjm krb5kdc[22410](info): commencing operation
krb5kdc: starting...
W20260620 23:12:25.586947 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:25 dist-test-slave-fzjm krb5kdc[22410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:25.693810 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:25.771385 22423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:25.771572 22423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:25.771596 22423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:25.772696 22423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:25.772740 22423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:25.772756 22423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:25.772769 22423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:25.772780 22423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:25.774156 22423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:25.774403 22423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:25.774607 22423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:12:25.779269 22423 server_base.cc:1034] running on GCE node
W20260620 23:12:25.779224 22429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:25.779241 22431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:25.779249 22428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:25.779701 22423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:25.780109 22423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:25.781253 22423 hybrid_clock.cc:648] HybridClock initialized: now 1781997145781235 us; error 34 us; skew 500 ppm
Jun 20 23:12:25 dist-test-slave-fzjm krb5kdc[22410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:25.783768 22423 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:25.784168 22423 webserver.cc:460] Webserver started at http://127.0.0.1:36181/ using document root <none> and password file <none>
I20260620 23:12:25.784386 22423 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:25.784428 22423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:25.784574 22423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:25.785557 22423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "ecb628bfa7974ed3b70a79ef6728e564"
format_stamp: "Formatted at 2026-06-20 23:12:25 on dist-test-slave-fzjm"
server_key: "9f001215891ac56cd4fa54965b8ecbff"
server_key_iv: "53a36b29e9c1a49bd072a616c694ffd4"
server_key_version: "encryptionkey@0"
I20260620 23:12:25.785928 22423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "ecb628bfa7974ed3b70a79ef6728e564"
format_stamp: "Formatted at 2026-06-20 23:12:25 on dist-test-slave-fzjm"
server_key: "9f001215891ac56cd4fa54965b8ecbff"
server_key_iv: "53a36b29e9c1a49bd072a616c694ffd4"
server_key_version: "encryptionkey@0"
I20260620 23:12:25.787300 22423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:25.788033 22438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:25.788241 22423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:25.788334 22423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "ecb628bfa7974ed3b70a79ef6728e564"
format_stamp: "Formatted at 2026-06-20 23:12:25 on dist-test-slave-fzjm"
server_key: "9f001215891ac56cd4fa54965b8ecbff"
server_key_iv: "53a36b29e9c1a49bd072a616c694ffd4"
server_key_version: "encryptionkey@0"
I20260620 23:12:25.788405 22423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:25.796128 22423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:25.798290 22423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:25.798432 22423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:25.802066 22423 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36087
I20260620 23:12:25.802093 22490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36087 every 8 connection(s)
I20260620 23:12:25.802397 22423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:25.803261 22491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:25.805429 22491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564: Bootstrap starting.
I20260620 23:12:25.806056 22491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:25.806355 22491 log.cc:824] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:25.807090 22491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564: No bootstrap required, opened a new log
I20260620 23:12:25.807828 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22423
I20260620 23:12:25.807946 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:25.808136 1153 external_mini_cluster.cc:1428] Setting key b52a383fa330ef46fed07ebc71a4e1d5
I20260620 23:12:25.809304 22491 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb628bfa7974ed3b70a79ef6728e564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36087 } }
I20260620 23:12:25.809428 22491 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:25.809449 22491 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb628bfa7974ed3b70a79ef6728e564, State: Initialized, Role: FOLLOWER
I20260620 23:12:25.809592 22491 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb628bfa7974ed3b70a79ef6728e564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36087 } }
I20260620 23:12:25.809664 22491 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:25.809703 22491 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:25.809788 22491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:25.810374 22491 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb628bfa7974ed3b70a79ef6728e564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36087 } }
I20260620 23:12:25.810456 22491 leader_election.cc:302] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecb628bfa7974ed3b70a79ef6728e564; no voters:
I20260620 23:12:25.810662 22491 leader_election.cc:288] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:25.810685 22496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:25.810801 22496 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [term 1 LEADER]: Becoming Leader. State: Replica: ecb628bfa7974ed3b70a79ef6728e564, State: Running, Role: LEADER
I20260620 23:12:25.810947 22491 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:25.810956 22496 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb628bfa7974ed3b70a79ef6728e564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36087 } }
I20260620 23:12:25.811475 22497 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecb628bfa7974ed3b70a79ef6728e564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb628bfa7974ed3b70a79ef6728e564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36087 } } }
I20260620 23:12:25.811717 22497 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:25.811641 22498 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecb628bfa7974ed3b70a79ef6728e564. Latest consensus state: current_term: 1 leader_uuid: "ecb628bfa7974ed3b70a79ef6728e564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb628bfa7974ed3b70a79ef6728e564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36087 } } }
I20260620 23:12:25.812001 22498 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:25.812481 22505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:25.812981 22505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:25.814824 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:25.808616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:40631 (local address 127.0.0.1:36087)
0620 23:12:25.808793 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:25.808800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:25.808823 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:12:25.809457 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:25.809465 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:25.809531 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:25.809746 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:25.809987 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:25.811107 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:25.812345 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:25.812947 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:25.813065 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:25.814141 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:25.814145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:25.814145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:25.814345 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:25.814373 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:25.814405 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:25.814491 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:25.814640 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 23:12:25.814673 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260620 23:12:25.815415 22505 catalog_manager.cc:1347] Generated new cluster ID: 938630de6139427bb1488ed567eba3b4
I20260620 23:12:25.815512 22505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:25.816205 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:25.825795 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:25.819174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:55881 (local address 127.0.0.1:36087)
0620 23:12:25.819208 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:25.819210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:25.823064 (+ 3854us) server_negotiation.cc:407] Connection header received
0620 23:12:25.823136 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:25.823137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:25.823146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:25.823226 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:25.823436 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:25.823890 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:25.824616 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:25.824839 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:25.824925 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:25.825553 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:25.825554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:25.825554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:25.825559 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:25.825572 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:25.825576 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:25.825628 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:25.825687 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 23:12:25.825696 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:12:25.855265 22505 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:25.855944 22505 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:25.861266 22505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecb628bfa7974ed3b70a79ef6728e564: Generated new TSK 0
I20260620 23:12:25.861439 22505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:25.871843 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22423
W20260620 23:12:25.876358 22408 connection.cc:535] client connection to 127.0.0.1:36087 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36087): Connection reset by peer (error 104)
2026-06-20T23:12:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:25 dist-test-slave-fzjm krb5kdc[22522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:25 dist-test-slave-fzjm krb5kdc[22522](info): set up 2 sockets
Jun 20 23:12:25 dist-test-slave-fzjm krb5kdc[22522](info): commencing operation
krb5kdc: starting...
W20260620 23:12:26.886653 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:26 dist-test-slave-fzjm krb5kdc[22522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:12:26.997473 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42597
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:27.076565 22535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:27.076799 22535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:27.076838 22535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:27.077957 22535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:27.078027 22535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:27.078053 22535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:27.078084 22535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:27.078115 22535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:27.079573 22535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42597
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:27.079845 22535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:27.080080 22535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:27.085040 22541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:27.085035 22543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:27.085196 22535 server_base.cc:1034] running on GCE node
W20260620 23:12:27.085059 22540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:27.085472 22535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:27.085757 22535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:27.086944 22535 hybrid_clock.cc:648] HybridClock initialized: now 1781997147086926 us; error 35 us; skew 500 ppm
Jun 20 23:12:27 dist-test-slave-fzjm krb5kdc[22522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:27.089279 22535 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:12:27.089682 22535 webserver.cc:460] Webserver started at http://127.0.0.1:33619/ using document root <none> and password file <none>
I20260620 23:12:27.089923 22535 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:27.089993 22535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:27.090137 22535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:27.091234 22535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "25d1e4a9d01040fe9881b52577d75c15"
format_stamp: "Formatted at 2026-06-20 23:12:27 on dist-test-slave-fzjm"
server_key: "d9fa3fc710d0286691675246da19c6ae"
server_key_iv: "d4962d8276ce337384019d87cd601c86"
server_key_version: "encryptionkey@0"
I20260620 23:12:27.091630 22535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "25d1e4a9d01040fe9881b52577d75c15"
format_stamp: "Formatted at 2026-06-20 23:12:27 on dist-test-slave-fzjm"
server_key: "d9fa3fc710d0286691675246da19c6ae"
server_key_iv: "d4962d8276ce337384019d87cd601c86"
server_key_version: "encryptionkey@0"
I20260620 23:12:27.092880 22535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:27.093556 22550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:27.093720 22535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:27.093828 22535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "25d1e4a9d01040fe9881b52577d75c15"
format_stamp: "Formatted at 2026-06-20 23:12:27 on dist-test-slave-fzjm"
server_key: "d9fa3fc710d0286691675246da19c6ae"
server_key_iv: "d4962d8276ce337384019d87cd601c86"
server_key_version: "encryptionkey@0"
I20260620 23:12:27.093901 22535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:27.097123 22535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:27.099048 22535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:27.099196 22535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:27.102979 22535 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42597
I20260620 23:12:27.103013 22602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42597 every 8 connection(s)
I20260620 23:12:27.103327 22535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:27.104261 22603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:27.106396 22603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15: Bootstrap starting.
I20260620 23:12:27.106997 22603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:27.107275 22603 log.cc:824] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:27.107909 22603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15: No bootstrap required, opened a new log
I20260620 23:12:27.109416 22603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d1e4a9d01040fe9881b52577d75c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260620 23:12:27.109547 22603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:27.109606 22603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d1e4a9d01040fe9881b52577d75c15, State: Initialized, Role: FOLLOWER
I20260620 23:12:27.109792 22603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d1e4a9d01040fe9881b52577d75c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260620 23:12:27.109874 22603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:27.109915 22603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:27.109962 22603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:27.110468 22603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d1e4a9d01040fe9881b52577d75c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260620 23:12:27.110579 22603 leader_election.cc:302] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25d1e4a9d01040fe9881b52577d75c15; no voters:
I20260620 23:12:27.110764 22603 leader_election.cc:288] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:27.110816 22606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:27.110992 22606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [term 1 LEADER]: Becoming Leader. State: Replica: 25d1e4a9d01040fe9881b52577d75c15, State: Running, Role: LEADER
I20260620 23:12:27.111037 22603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:27.111156 22606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d1e4a9d01040fe9881b52577d75c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260620 23:12:27.111526 22607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25d1e4a9d01040fe9881b52577d75c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d1e4a9d01040fe9881b52577d75c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } } }
I20260620 23:12:27.111663 22607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:27.111699 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22535
I20260620 23:12:27.111809 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:27.112016 1153 external_mini_cluster.cc:1428] Setting key f3d015ed3afa024cbb4d786cf033ec84
I20260620 23:12:27.112217 22610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:27.112720 22610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:27.111541 22608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25d1e4a9d01040fe9881b52577d75c15. Latest consensus state: current_term: 1 leader_uuid: "25d1e4a9d01040fe9881b52577d75c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d1e4a9d01040fe9881b52577d75c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } } }
I20260620 23:12:27.112888 22608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:27.114403 22610 catalog_manager.cc:1347] Generated new cluster ID: e8052d00a6be49f0926ab8d21b69192f
I20260620 23:12:27.114454 22610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:27.114667 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:27.112620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:46571 (local address 127.0.0.1:42597)
0620 23:12:27.113006 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:27.113015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:27.113051 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:12:27.113161 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:27.113169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:27.113304 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:27.113438 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:27.114226 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:27.114229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:27.114230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:27.114234 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:27.114284 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:27.114299 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:27.114361 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:27.114461 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 23:12:27.114505 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":62,"threads_started":1}
I20260620 23:12:27.115481 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:27.128159 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:27.116863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.6:36465 (local address 127.0.0.1:42597)
0620 23:12:27.116892 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:27.116894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:27.127134 (+ 10240us) server_negotiation.cc:407] Connection header received
0620 23:12:27.127213 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:27.127214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:27.127220 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:27.127279 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:27.127914 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:27.127915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:27.127915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:27.127919 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:27.127935 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:27.127940 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:27.127993 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:27.128053 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 23:12:27.128067 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:12:27.147444 22610 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:27.147673 22610 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:27.155072 22610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25d1e4a9d01040fe9881b52577d75c15: Generated new TSK 0
I20260620 23:12:27.155292 22610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:27.157724 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22535
2026-06-20T23:12:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:27 dist-test-slave-fzjm krb5kdc[22634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:27 dist-test-slave-fzjm krb5kdc[22634](info): set up 2 sockets
Jun 20 23:12:27 dist-test-slave-fzjm krb5kdc[22634](info): commencing operation
krb5kdc: starting...
W20260620 23:12:28.179247 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:28 dist-test-slave-fzjm krb5kdc[22634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:28.290231 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36185
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:36185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:28.369143 22647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:28.369350 22647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:28.369382 22647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:28.370426 22647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:28.370491 22647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:28.370512 22647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:28.370532 22647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:28.370558 22647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:28.371974 22647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:36185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:28.372236 22647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:28.372437 22647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:28.377128 22653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:28.377151 22655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:28.377137 22652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:28.377278 22647 server_base.cc:1034] running on GCE node
I20260620 23:12:28.377504 22647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:28.377810 22647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:28.378942 22647 hybrid_clock.cc:648] HybridClock initialized: now 1781997148378933 us; error 34 us; skew 500 ppm
Jun 20 23:12:28 dist-test-slave-fzjm krb5kdc[22634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997148, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:28.381289 22647 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:28.381722 22647 webserver.cc:460] Webserver started at http://127.1.32.126:44453/ using document root <none> and password file <none>
I20260620 23:12:28.381937 22647 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:28.381997 22647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:28.382167 22647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:28.383272 22647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "c305e4fb5a394844a001ad0fb56587af"
format_stamp: "Formatted at 2026-06-20 23:12:28 on dist-test-slave-fzjm"
server_key: "9a3fcd762b741a1ba7134baed158b22b"
server_key_iv: "f7dbba2901b7337635ff143c3f111671"
server_key_version: "encryptionkey@0"
I20260620 23:12:28.383607 22647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "c305e4fb5a394844a001ad0fb56587af"
format_stamp: "Formatted at 2026-06-20 23:12:28 on dist-test-slave-fzjm"
server_key: "9a3fcd762b741a1ba7134baed158b22b"
server_key_iv: "f7dbba2901b7337635ff143c3f111671"
server_key_version: "encryptionkey@0"
I20260620 23:12:28.384860 22647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:28.385524 22662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:28.385684 22647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:28.385779 22647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "c305e4fb5a394844a001ad0fb56587af"
format_stamp: "Formatted at 2026-06-20 23:12:28 on dist-test-slave-fzjm"
server_key: "9a3fcd762b741a1ba7134baed158b22b"
server_key_iv: "f7dbba2901b7337635ff143c3f111671"
server_key_version: "encryptionkey@0"
I20260620 23:12:28.385854 22647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:28.405897 22647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:28.408041 22647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:28.408207 22647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:28.412037 22647 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:36185
I20260620 23:12:28.412089 22714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:36185 every 8 connection(s)
I20260620 23:12:28.412472 22647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:28.413329 22715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:28.414377 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22647
I20260620 23:12:28.414520 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:28.414702 1153 external_mini_cluster.cc:1428] Setting key b015e75c015e30318d396184fb729801
I20260620 23:12:28.416051 22715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af: Bootstrap starting.
I20260620 23:12:28.416646 22715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:28.416960 22715 log.cc:824] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:28.417658 22715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af: No bootstrap required, opened a new log
I20260620 23:12:28.419761 22715 raft_consensus.cc:348] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c305e4fb5a394844a001ad0fb56587af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36185 } }
I20260620 23:12:28.419919 22715 raft_consensus.cc:374] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:28.419988 22715 raft_consensus.cc:729] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c305e4fb5a394844a001ad0fb56587af, State: Initialized, Role: FOLLOWER
I20260620 23:12:28.420141 22715 consensus_queue.cc:260] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c305e4fb5a394844a001ad0fb56587af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36185 } }
I20260620 23:12:28.420243 22715 raft_consensus.cc:388] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:28.420286 22715 raft_consensus.cc:482] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:28.420334 22715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:28.420835 22715 raft_consensus.cc:504] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c305e4fb5a394844a001ad0fb56587af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36185 } }
I20260620 23:12:28.420948 22715 leader_election.cc:302] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c305e4fb5a394844a001ad0fb56587af; no voters:
I20260620 23:12:28.421116 22715 leader_election.cc:288] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:28.421169 22720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:28.421330 22720 raft_consensus.cc:686] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [term 1 LEADER]: Becoming Leader. State: Replica: c305e4fb5a394844a001ad0fb56587af, State: Running, Role: LEADER
I20260620 23:12:28.421376 22715 sys_catalog.cc:564] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:28.421509 22720 consensus_queue.cc:237] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c305e4fb5a394844a001ad0fb56587af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36185 } }
Jun 20 23:12:28 dist-test-slave-fzjm krb5kdc[22634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:28.421877 22720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [sys.catalog]: SysCatalogTable state changed. Reason: New leader c305e4fb5a394844a001ad0fb56587af. Latest consensus state: current_term: 1 leader_uuid: "c305e4fb5a394844a001ad0fb56587af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c305e4fb5a394844a001ad0fb56587af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36185 } } }
I20260620 23:12:28.421949 22720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:28.421978 22721 sys_catalog.cc:455] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c305e4fb5a394844a001ad0fb56587af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c305e4fb5a394844a001ad0fb56587af" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36185 } } }
I20260620 23:12:28.422060 22721 sys_catalog.cc:458] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af [sys.catalog]: This master's current role is: LEADER
W20260620 23:12:28.423323 22736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:28.423671 22736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:28.423733 22726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:28.423902 22726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:28.425429 22726 catalog_manager.cc:1347] Generated new cluster ID: dcf6512609044edb8751b07d2d88d5ee
I20260620 23:12:28.425479 22726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:28.426378 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:28.415214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49845 (local address 127.1.32.126:36185)
0620 23:12:28.415395 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:28.415405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:28.415438 (+ 33us) server_negotiation.cc:407] Connection header received
0620 23:12:28.415969 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:28.415978 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:28.416037 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:28.416188 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:28.416815 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:28.418117 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:28.418819 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:28.419507 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:28.419614 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:28.422263 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:28.422272 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:28.422273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:28.422283 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:28.424408 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:28.424816 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:28.424819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:28.424819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:28.424927 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:28.425171 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:28.425172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:28.425172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:28.425458 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:28.425610 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:28.426078 (+ 468us) server_negotiation.cc:299] Negotiation successful
0620 23:12:28.426167 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260620 23:12:28.427265 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:28.435985 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:28.428838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48939 (local address 127.1.32.126:36185)
0620 23:12:28.428866 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:28.428868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:28.428881 (+ 13us) server_negotiation.cc:407] Connection header received
0620 23:12:28.428954 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:28.428955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:28.428963 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:28.429009 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:28.429639 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:28.430351 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:28.431371 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:28.431666 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:28.431750 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:28.433165 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:28.433166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:28.433166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:28.433172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:28.434671 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:28.435089 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:28.435089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:28.435090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:28.435158 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:28.435402 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:28.435403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:28.435403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:28.435513 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:28.435601 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:28.435759 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 23:12:28.435815 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260620 23:12:28.452762 22726 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:28.453519 22726 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:28.460145 22726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c305e4fb5a394844a001ad0fb56587af: Generated new TSK 0
I20260620 23:12:28.460348 22726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:28.468770 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22647
W20260620 23:12:28.472584 22632 connection.cc:535] client connection to 127.1.32.126:36185 recv error: Network error: failed to read from TLS socket (remote: 127.1.32.126:36185): Connection reset by peer (error 104)
2026-06-20T23:12:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:28 dist-test-slave-fzjm krb5kdc[22746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:28 dist-test-slave-fzjm krb5kdc[22746](info): set up 2 sockets
Jun 20 23:12:28 dist-test-slave-fzjm krb5kdc[22746](info): commencing operation
krb5kdc: starting...
W20260620 23:12:29.507006 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:29 dist-test-slave-fzjm krb5kdc[22746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:29.617399 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33627
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:38419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:33627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:29.694455 22759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:29.694629 22759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:29.694648 22759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:29.695641 22759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:29.695695 22759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:29.695708 22759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:29.695721 22759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:29.695732 22759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:29.697077 22759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:38419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:33627
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:29.697271 22759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:29.697419 22759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:29.701937 22765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:29.701988 22764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:29.701937 22767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:29.702154 22759 server_base.cc:1034] running on GCE node
I20260620 23:12:29.702346 22759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:29.702628 22759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:29.703783 22759 hybrid_clock.cc:648] HybridClock initialized: now 1781997149703758 us; error 28 us; skew 500 ppm
Jun 20 23:12:29 dist-test-slave-fzjm krb5kdc[22746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997149, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:29.710371 22759 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:29.710829 22759 webserver.cc:460] Webserver started at http://127.1.32.126:40099/ using document root <none> and password file <none>
I20260620 23:12:29.711071 22759 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:29.711139 22759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:29.711285 22759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:29.712292 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "cc7386a437094b01b6ec93702d66a7a3"
format_stamp: "Formatted at 2026-06-20 23:12:29 on dist-test-slave-fzjm"
server_key: "6e80a3e6e11ae4b57e08e204438ea648"
server_key_iv: "6c6354a0dd50227e8c91e1e8c3e0b896"
server_key_version: "encryptionkey@0"
I20260620 23:12:29.712687 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "cc7386a437094b01b6ec93702d66a7a3"
format_stamp: "Formatted at 2026-06-20 23:12:29 on dist-test-slave-fzjm"
server_key: "6e80a3e6e11ae4b57e08e204438ea648"
server_key_iv: "6c6354a0dd50227e8c91e1e8c3e0b896"
server_key_version: "encryptionkey@0"
I20260620 23:12:29.714484 22759 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 23:12:29.715173 22774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:29.715344 22759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 23:12:29.715440 22759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "cc7386a437094b01b6ec93702d66a7a3"
format_stamp: "Formatted at 2026-06-20 23:12:29 on dist-test-slave-fzjm"
server_key: "6e80a3e6e11ae4b57e08e204438ea648"
server_key_iv: "6c6354a0dd50227e8c91e1e8c3e0b896"
server_key_version: "encryptionkey@0"
I20260620 23:12:29.715523 22759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:29.733166 22759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:29.734932 22759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:29.735091 22759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:29.739028 22759 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:33627
I20260620 23:12:29.739075 22826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:33627 every 8 connection(s)
I20260620 23:12:29.739413 22759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:29.740252 22827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:29.741746 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22759
I20260620 23:12:29.741868 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:29.742066 1153 external_mini_cluster.cc:1428] Setting key 44aa89cccb30ce9f5422c82e69a48c62
I20260620 23:12:29.742786 22827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3: Bootstrap starting.
I20260620 23:12:29.743438 22827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:29.743765 22827 log.cc:824] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:29.744439 22827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3: No bootstrap required, opened a new log
I20260620 23:12:29.746765 22827 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7386a437094b01b6ec93702d66a7a3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33627 } }
I20260620 23:12:29.746959 22827 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:29.747033 22827 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7386a437094b01b6ec93702d66a7a3, State: Initialized, Role: FOLLOWER
I20260620 23:12:29.747159 22827 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7386a437094b01b6ec93702d66a7a3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33627 } }
I20260620 23:12:29.747221 22827 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:29.747265 22827 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:29.747311 22827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:29.747692 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:29.742556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38171 (local address 127.1.32.126:33627)
0620 23:12:29.742721 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:29.742729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:29.742763 (+ 34us) server_negotiation.cc:407] Connection header received
0620 23:12:29.743256 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:29.743270 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:29.743318 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:29.743437 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:29.743691 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:29.744664 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:29.745714 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:29.746087 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:29.746170 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:29.746925 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:29.746929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:29.746929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:29.747186 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:29.747227 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:29.747242 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:29.747332 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:29.747452 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 23:12:29.747497 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260620 23:12:29.747911 22827 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7386a437094b01b6ec93702d66a7a3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33627 } }
I20260620 23:12:29.748018 22827 leader_election.cc:302] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc7386a437094b01b6ec93702d66a7a3; no voters:
I20260620 23:12:29.748180 22827 leader_election.cc:288] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:29.748229 22832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:29.748368 22832 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [term 1 LEADER]: Becoming Leader. State: Replica: cc7386a437094b01b6ec93702d66a7a3, State: Running, Role: LEADER
I20260620 23:12:29.748417 22827 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:29.748529 22832 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7386a437094b01b6ec93702d66a7a3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33627 } }
I20260620 23:12:29.748893 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:29.749603 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc7386a437094b01b6ec93702d66a7a3. Latest consensus state: current_term: 1 leader_uuid: "cc7386a437094b01b6ec93702d66a7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7386a437094b01b6ec93702d66a7a3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33627 } } }
I20260620 23:12:29.749693 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:29.749868 22847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:29.750016 22834 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc7386a437094b01b6ec93702d66a7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7386a437094b01b6ec93702d66a7a3" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33627 } } }
I20260620 23:12:29.750077 22834 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:29.750422 22847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:29.751793 22847 catalog_manager.cc:1347] Generated new cluster ID: d6c985afb1cf4d8388340a1da595422b
I20260620 23:12:29.751859 22847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:29.754091 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:29.750401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44559 (local address 127.1.32.126:33627)
0620 23:12:29.750437 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:29.750439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:29.750644 (+ 205us) server_negotiation.cc:407] Connection header received
0620 23:12:29.750746 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:29.750747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:29.750758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:29.750787 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:29.751021 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:29.751708 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:29.752703 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:29.752991 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:29.753032 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:29.753828 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:29.753830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:29.753830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:29.753835 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:29.753849 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:29.753853 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:29.753884 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:29.754013 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 23:12:29.754024 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:12:29.757177 22847 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:29.757656 22847 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:29.762830 22847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc7386a437094b01b6ec93702d66a7a3: Generated new TSK 0
I20260620 23:12:29.763022 22847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:29.765394 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22759
W20260620 23:12:29.769569 22744 connection.cc:535] client connection to 127.1.32.126:33627 recv error: Network error: failed to read from TLS socket (remote: 127.1.32.126:33627): Connection reset by peer (error 104)
2026-06-20T23:12:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:29 dist-test-slave-fzjm krb5kdc[22858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:29 dist-test-slave-fzjm krb5kdc[22858](info): set up 2 sockets
Jun 20 23:12:29 dist-test-slave-fzjm krb5kdc[22858](info): commencing operation
krb5kdc: starting...
W20260620 23:12:30.771116 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:30 dist-test-slave-fzjm krb5kdc[22858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:30.881219 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36797
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:36797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:30.958074 22871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:30.958333 22871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:30.958367 22871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:30.959538 22871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:30.959607 22871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:30.959630 22871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:30.959649 22871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:30.959679 22871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:30.961108 22871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:36797
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:30.961421 22871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:30.961644 22871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:30.966300 22879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:30.966310 22877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:30.966341 22871 server_base.cc:1034] running on GCE node
W20260620 23:12:30.966375 22876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:30.966972 22871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:30.967265 22871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:30.968438 22871 hybrid_clock.cc:648] HybridClock initialized: now 1781997150968430 us; error 38 us; skew 500 ppm
Jun 20 23:12:30 dist-test-slave-fzjm krb5kdc[22858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997150, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:30.970989 22871 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:30.971462 22871 webserver.cc:460] Webserver started at http://127.1.32.126:37661/ using document root <none> and password file <none>
I20260620 23:12:30.971738 22871 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:30.971814 22871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:30.971946 22871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:30.973038 22871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604"
format_stamp: "Formatted at 2026-06-20 23:12:30 on dist-test-slave-fzjm"
server_key: "41d7e4f47537f91ff950b31dc258fb22"
server_key_iv: "6990d20e13dd9eb91dddf37a7cc12bb2"
server_key_version: "encryptionkey@0"
I20260620 23:12:30.973398 22871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604"
format_stamp: "Formatted at 2026-06-20 23:12:30 on dist-test-slave-fzjm"
server_key: "41d7e4f47537f91ff950b31dc258fb22"
server_key_iv: "6990d20e13dd9eb91dddf37a7cc12bb2"
server_key_version: "encryptionkey@0"
I20260620 23:12:30.974660 22871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:30.975423 22886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:30.975615 22871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:30.975725 22871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604"
format_stamp: "Formatted at 2026-06-20 23:12:30 on dist-test-slave-fzjm"
server_key: "41d7e4f47537f91ff950b31dc258fb22"
server_key_iv: "6990d20e13dd9eb91dddf37a7cc12bb2"
server_key_version: "encryptionkey@0"
I20260620 23:12:30.975841 22871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:30.979074 22871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:30.981271 22871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:30.981434 22871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:30.985322 22871 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:36797
I20260620 23:12:30.985348 22938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:36797 every 8 connection(s)
I20260620 23:12:30.985651 22871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:30.986505 22939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:30.988548 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604: Bootstrap starting.
I20260620 23:12:30.989030 22939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:30.989229 22939 log.cc:824] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:30.989780 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604: No bootstrap required, opened a new log
I20260620 23:12:30.991237 22939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36797 } }
I20260620 23:12:30.991359 22939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:30.991379 22939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ddaf2fc3bdd4a3eb9115d9fb408c604, State: Initialized, Role: FOLLOWER
I20260620 23:12:30.991464 22939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36797 } }
I20260620 23:12:30.991509 22939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:30.991528 22939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:30.991587 22939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:30.992332 22939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36797 } }
I20260620 23:12:30.992434 22939 leader_election.cc:302] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ddaf2fc3bdd4a3eb9115d9fb408c604; no voters:
I20260620 23:12:30.992616 22939 leader_election.cc:288] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:30.992694 22942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:30.992887 22939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:30.992908 22942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [term 1 LEADER]: Becoming Leader. State: Replica: 3ddaf2fc3bdd4a3eb9115d9fb408c604, State: Running, Role: LEADER
I20260620 23:12:30.992983 22942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36797 } }
I20260620 23:12:30.993372 22943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36797 } } }
I20260620 23:12:30.993383 22944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ddaf2fc3bdd4a3eb9115d9fb408c604. Latest consensus state: current_term: 1 leader_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddaf2fc3bdd4a3eb9115d9fb408c604" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 36797 } } }
I20260620 23:12:30.993554 22944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:30.993541 22943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:30.994041 22952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:30.994521 22952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:30.995329 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22871
I20260620 23:12:30.995469 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:30.995654 1153 external_mini_cluster.cc:1428] Setting key 6bfdcede5f1dd335d37a9937e872d108
I20260620 23:12:30.997681 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:30.996130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38633 (local address 127.1.32.126:36797)
0620 23:12:30.996298 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:30.996307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:30.996339 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:12:30.996449 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:30.996454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:30.996493 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:30.996621 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:30.997218 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:30.997221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:30.997222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:30.997225 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:30.997273 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:30.997283 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:30.997345 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:30.997425 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:12:30.997559 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260620 23:12:30.998618 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:30.999956 22952 catalog_manager.cc:1347] Generated new cluster ID: bd9316f7b8724896b92cfef104d27ef4
I20260620 23:12:31.000067 22952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:31.001147 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:31.000097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56075 (local address 127.1.32.126:36797)
0620 23:12:31.000133 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:31.000134 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:31.000283 (+ 149us) server_negotiation.cc:407] Connection header received
0620 23:12:31.000339 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:31.000340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:31.000344 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:31.000396 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:31.000950 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:31.000951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:31.000952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:31.000956 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:31.000967 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:31.000971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:31.001025 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:31.001077 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 23:12:31.001089 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:31.019788 22952 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:31.019943 22952 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:31.036916 22952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ddaf2fc3bdd4a3eb9115d9fb408c604: Generated new TSK 0
I20260620 23:12:31.037120 22952 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:31.045001 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22871
2026-06-20T23:12:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:31 dist-test-slave-fzjm krb5kdc[22970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:31 dist-test-slave-fzjm krb5kdc[22970](info): set up 2 sockets
Jun 20 23:12:31 dist-test-slave-fzjm krb5kdc[22970](info): commencing operation
krb5kdc: starting...
W20260620 23:12:32.058835 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:32 dist-test-slave-fzjm krb5kdc[22970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:32.172137 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43219
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:42057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:43219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:32.252657 22983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:32.252933 22983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:32.252972 22983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:32.254149 22983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:32.254231 22983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:32.254259 22983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:32.254294 22983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:32.254329 22983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:32.255832 22983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:42057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:43219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:43219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:32.256125 22983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:32.256368 22983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:32.261188 22991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:32.261178 22989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:32.261387 22983 server_base.cc:1034] running on GCE node
W20260620 23:12:32.261176 22988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:32.261698 22983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:32.262006 22983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:32.263202 22983 hybrid_clock.cc:648] HybridClock initialized: now 1781997152263192 us; error 38 us; skew 500 ppm
Jun 20 23:12:32 dist-test-slave-fzjm krb5kdc[22970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997152, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:32.265825 22983 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:32.266278 22983 webserver.cc:460] Webserver started at http://127.1.32.126:41041/ using document root <none> and password file <none>
I20260620 23:12:32.266556 22983 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:32.266634 22983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:32.266808 22983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:32.267938 22983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "2e4e44a8f5884770bd42735e3aa84b1c"
format_stamp: "Formatted at 2026-06-20 23:12:32 on dist-test-slave-fzjm"
server_key: "038a0dccc603b19ab0381a14fdc41707"
server_key_iv: "e1ee1310722b1c7c0e54ab93c7ff1ebd"
server_key_version: "encryptionkey@0"
I20260620 23:12:32.268329 22983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "2e4e44a8f5884770bd42735e3aa84b1c"
format_stamp: "Formatted at 2026-06-20 23:12:32 on dist-test-slave-fzjm"
server_key: "038a0dccc603b19ab0381a14fdc41707"
server_key_iv: "e1ee1310722b1c7c0e54ab93c7ff1ebd"
server_key_version: "encryptionkey@0"
I20260620 23:12:32.269709 22983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:32.270444 22998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:32.270614 22983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:32.270718 22983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "2e4e44a8f5884770bd42735e3aa84b1c"
format_stamp: "Formatted at 2026-06-20 23:12:32 on dist-test-slave-fzjm"
server_key: "038a0dccc603b19ab0381a14fdc41707"
server_key_iv: "e1ee1310722b1c7c0e54ab93c7ff1ebd"
server_key_version: "encryptionkey@0"
I20260620 23:12:32.270804 22983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:32.281455 22983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:32.283663 22983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:32.283890 22983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:32.288070 22983 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:43219
I20260620 23:12:32.288117 23050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:43219 every 8 connection(s)
I20260620 23:12:32.288481 22983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:32.289327 23051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:32.291622 23051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c: Bootstrap starting.
I20260620 23:12:32.292210 23051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:32.292503 23051 log.cc:824] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:32.293237 23051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c: No bootstrap required, opened a new log
I20260620 23:12:32.294859 23051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43219 } }
I20260620 23:12:32.295058 23051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:32.295084 23051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4e44a8f5884770bd42735e3aa84b1c, State: Initialized, Role: FOLLOWER
I20260620 23:12:32.295256 23051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43219 } }
I20260620 23:12:32.295318 23051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:32.295339 23051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:32.295410 23051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:32.296247 23051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43219 } }
I20260620 23:12:32.296334 23051 leader_election.cc:302] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e4e44a8f5884770bd42735e3aa84b1c; no voters:
I20260620 23:12:32.296523 23051 leader_election.cc:288] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:32.296571 23054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:32.296749 23054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [term 1 LEADER]: Becoming Leader. State: Replica: 2e4e44a8f5884770bd42735e3aa84b1c, State: Running, Role: LEADER
I20260620 23:12:32.297008 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 22983
I20260620 23:12:32.297124 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:32.297318 1153 external_mini_cluster.cc:1428] Setting key 29a027e6ec299bb09a12303ed7ee3d2d
I20260620 23:12:32.296804 23051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:32.297495 23054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43219 } }
I20260620 23:12:32.298058 23055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43219 } } }
I20260620 23:12:32.298118 23056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e4e44a8f5884770bd42735e3aa84b1c. Latest consensus state: current_term: 1 leader_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e44a8f5884770bd42735e3aa84b1c" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 43219 } } }
I20260620 23:12:32.298247 23056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:32.298172 23055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:32.298430 23065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:32.299952 23065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:32.301957 23065 catalog_manager.cc:1347] Generated new cluster ID: cab417982b174a4ea66f0aab046558e1
I20260620 23:12:32.302013 23065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 23:12:32 dist-test-slave-fzjm krb5kdc[22970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:32.309340 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:32.297912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47647 (local address 127.1.32.126:43219)
0620 23:12:32.298366 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:32.298376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:32.298413 (+ 37us) server_negotiation.cc:407] Connection header received
0620 23:12:32.298536 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:32.298543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:32.298608 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:32.298760 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:32.299552 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:32.300921 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:32.301900 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:32.302498 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:32.302602 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:32.305486 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:32.305491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:32.305491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:32.305503 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:32.307670 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:32.308050 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:32.308053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:32.308054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:32.308182 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:32.308394 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:32.308395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:32.308396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:32.308715 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:32.308883 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:32.308951 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 23:12:32.309095 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":68,"threads_started":1}
I20260620 23:12:32.310295 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:32.312919 23065 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:32.313545 23065 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:32.318917 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:32.311769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47699 (local address 127.1.32.126:43219)
0620 23:12:32.311802 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:32.311804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:32.311875 (+ 71us) server_negotiation.cc:407] Connection header received
0620 23:12:32.311930 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:32.311931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:32.311940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:32.311992 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:32.312783 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:32.313517 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:32.314517 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:32.314838 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:32.314935 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:32.316191 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:32.316193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:32.316193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:32.316200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:32.317725 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:32.318104 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:32.318105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:32.318105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:32.318161 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:32.318366 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:32.318366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:32.318367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:32.318492 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:32.318613 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:32.318718 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 23:12:32.318805 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:12:32.319761 23065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e4e44a8f5884770bd42735e3aa84b1c: Generated new TSK 0
I20260620 23:12:32.319923 23065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:32.323509 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 22983
2026-06-20T23:12:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:32 dist-test-slave-fzjm krb5kdc[23082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:32 dist-test-slave-fzjm krb5kdc[23082](info): set up 2 sockets
Jun 20 23:12:32 dist-test-slave-fzjm krb5kdc[23082](info): commencing operation
krb5kdc: starting...
W20260620 23:12:33.343111 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:33 dist-test-slave-fzjm krb5kdc[23082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:33.459583 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33149
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:33149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:33.539320 23095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:33.539551 23095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:33.539584 23095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:33.540707 23095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:33.540786 23095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:33.540812 23095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:33.540840 23095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:33.540863 23095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:33.542341 23095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:33149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:33149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:33.542630 23095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:33.542876 23095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:33.547675 23101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:33.547654 23103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:33.547689 23095 server_base.cc:1034] running on GCE node
W20260620 23:12:33.547653 23100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:33.548101 23095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:33.548436 23095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:33.549607 23095 hybrid_clock.cc:648] HybridClock initialized: now 1781997153549592 us; error 29 us; skew 500 ppm
Jun 20 23:12:33 dist-test-slave-fzjm krb5kdc[23082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997153, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:33.552050 23095 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:33.552460 23095 webserver.cc:460] Webserver started at http://127.1.32.126:44251/ using document root <none> and password file <none>
I20260620 23:12:33.552713 23095 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:33.552758 23095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:33.552901 23095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:33.554015 23095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "5bd976a92247408b95749b3efd79b316"
format_stamp: "Formatted at 2026-06-20 23:12:33 on dist-test-slave-fzjm"
server_key: "4b5b031352a5eefaa70d432a8f0d0f7a"
server_key_iv: "065900fc814d1c4c76129134b15288ff"
server_key_version: "encryptionkey@0"
I20260620 23:12:33.554382 23095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "5bd976a92247408b95749b3efd79b316"
format_stamp: "Formatted at 2026-06-20 23:12:33 on dist-test-slave-fzjm"
server_key: "4b5b031352a5eefaa70d432a8f0d0f7a"
server_key_iv: "065900fc814d1c4c76129134b15288ff"
server_key_version: "encryptionkey@0"
I20260620 23:12:33.555732 23095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:33.556450 23110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:33.556677 23095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 23:12:33.556752 23095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "5bd976a92247408b95749b3efd79b316"
format_stamp: "Formatted at 2026-06-20 23:12:33 on dist-test-slave-fzjm"
server_key: "4b5b031352a5eefaa70d432a8f0d0f7a"
server_key_iv: "065900fc814d1c4c76129134b15288ff"
server_key_version: "encryptionkey@0"
I20260620 23:12:33.556834 23095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:33.568738 23095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:33.570811 23095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:33.571031 23095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:33.574990 23095 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:33149
I20260620 23:12:33.575014 23162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:33149 every 8 connection(s)
I20260620 23:12:33.575395 23095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:33.576233 23163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:33.578658 23163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316: Bootstrap starting.
I20260620 23:12:33.579329 23163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:33.579615 23163 log.cc:824] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:33.580322 23163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316: No bootstrap required, opened a new log
I20260620 23:12:33.581928 23163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd976a92247408b95749b3efd79b316" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33149 } }
I20260620 23:12:33.582070 23163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:33.582093 23163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd976a92247408b95749b3efd79b316, State: Initialized, Role: FOLLOWER
I20260620 23:12:33.582255 23163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd976a92247408b95749b3efd79b316" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33149 } }
I20260620 23:12:33.582324 23163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:33.582346 23163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:33.582399 23163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:33.583334 23163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd976a92247408b95749b3efd79b316" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33149 } }
I20260620 23:12:33.583424 23163 leader_election.cc:302] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bd976a92247408b95749b3efd79b316; no voters:
I20260620 23:12:33.583627 23163 leader_election.cc:288] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:33.583681 23166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:33.583921 23166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [term 1 LEADER]: Becoming Leader. State: Replica: 5bd976a92247408b95749b3efd79b316, State: Running, Role: LEADER
I20260620 23:12:33.583971 23163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:33.584012 23166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd976a92247408b95749b3efd79b316" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33149 } }
I20260620 23:12:33.584313 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23095
I20260620 23:12:33.584434 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:33.584631 1153 external_mini_cluster.cc:1428] Setting key 61712939788fc4d08d276900a5272550
I20260620 23:12:33.585172 23167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bd976a92247408b95749b3efd79b316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd976a92247408b95749b3efd79b316" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33149 } } }
I20260620 23:12:33.585265 23167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:33.585511 23168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bd976a92247408b95749b3efd79b316. Latest consensus state: current_term: 1 leader_uuid: "5bd976a92247408b95749b3efd79b316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd976a92247408b95749b3efd79b316" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 33149 } } }
I20260620 23:12:33.585590 23168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:33.585448 23183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:33.586359 23183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:33.587829 23183 catalog_manager.cc:1347] Generated new cluster ID: 868478dda3074399a84aef543edc4a1d
I20260620 23:12:33.587879 23183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:33.591061 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:33.585252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48121 (local address 127.1.32.126:33149)
0620 23:12:33.585618 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:33.585628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:33.585666 (+ 38us) server_negotiation.cc:407] Connection header received
0620 23:12:33.585789 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:33.585796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:33.585854 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:33.586009 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:33.586357 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:33.587774 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:33.588927 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:33.589516 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:33.589617 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:33.590381 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:33.590385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:33.590385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:33.590578 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:33.590614 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:33.590628 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:33.590695 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:33.590787 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 23:12:33.590845 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":78,"threads_started":1}
I20260620 23:12:33.591936 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:33.596827 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:33.593336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47951 (local address 127.1.32.126:33149)
0620 23:12:33.593367 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:33.593368 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:33.593543 (+ 175us) server_negotiation.cc:407] Connection header received
0620 23:12:33.593624 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:33.593625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:33.593634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:33.593660 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:33.593840 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:33.594560 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:33.595560 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:33.595857 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:33.595890 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:33.596591 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:33.596592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:33.596592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:33.596596 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:33.596608 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:33.596612 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:33.596636 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:33.596735 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 23:12:33.596758 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 23:12:33.601433 23183 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:33.601924 23183 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:33.609799 23183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bd976a92247408b95749b3efd79b316: Generated new TSK 0
I20260620 23:12:33.609998 23183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:33.614177 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23095
2026-06-20T23:12:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:33 dist-test-slave-fzjm krb5kdc[23194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:33 dist-test-slave-fzjm krb5kdc[23194](info): set up 2 sockets
Jun 20 23:12:33 dist-test-slave-fzjm krb5kdc[23194](info): commencing operation
krb5kdc: starting...
W20260620 23:12:34.635057 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:34 dist-test-slave-fzjm krb5kdc[23194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:34.745971 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46727
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:40875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:46727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:34.826581 23207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:34.826854 23207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:34.826894 23207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:34.828137 23207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:34.828219 23207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:34.828248 23207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:34.828280 23207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:34.828313 23207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:34.829835 23207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:40875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:46727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:46727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:34.830120 23207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:34.830374 23207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:34.835309 23213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:34.835392 23212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:34.835304 23215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:34.835399 23207 server_base.cc:1034] running on GCE node
I20260620 23:12:34.835778 23207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:34.836081 23207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:34.837249 23207 hybrid_clock.cc:648] HybridClock initialized: now 1781997154837236 us; error 26 us; skew 500 ppm
Jun 20 23:12:34 dist-test-slave-fzjm krb5kdc[23194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997154, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:34.842656 23207 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:34.843181 23207 webserver.cc:460] Webserver started at http://127.1.32.126:41865/ using document root <none> and password file <none>
I20260620 23:12:34.843405 23207 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:34.843472 23207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:34.843617 23207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:34.844664 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "5b77de12ef7b478e89af4bbfc1139142"
format_stamp: "Formatted at 2026-06-20 23:12:34 on dist-test-slave-fzjm"
server_key: "78afca30571cc6e666dc7d9861349b94"
server_key_iv: "1abde4980226a23a9fde192115568731"
server_key_version: "encryptionkey@0"
I20260620 23:12:34.845031 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "5b77de12ef7b478e89af4bbfc1139142"
format_stamp: "Formatted at 2026-06-20 23:12:34 on dist-test-slave-fzjm"
server_key: "78afca30571cc6e666dc7d9861349b94"
server_key_iv: "1abde4980226a23a9fde192115568731"
server_key_version: "encryptionkey@0"
I20260620 23:12:34.846297 23207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:34.847031 23222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:34.847201 23207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:34.847295 23207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "5b77de12ef7b478e89af4bbfc1139142"
format_stamp: "Formatted at 2026-06-20 23:12:34 on dist-test-slave-fzjm"
server_key: "78afca30571cc6e666dc7d9861349b94"
server_key_iv: "1abde4980226a23a9fde192115568731"
server_key_version: "encryptionkey@0"
I20260620 23:12:34.847375 23207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:34.850615 23207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:34.852939 23207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:34.853098 23207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:34.857122 23207 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:46727
I20260620 23:12:34.857187 23274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:46727 every 8 connection(s)
I20260620 23:12:34.857504 23207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:34.858469 23275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:34.860441 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23207
I20260620 23:12:34.860559 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:34.860769 1153 external_mini_cluster.cc:1428] Setting key 5285e01a7d36eccc4cf657b24b1eb1be
I20260620 23:12:34.861002 23275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142: Bootstrap starting.
I20260620 23:12:34.861673 23275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:34.861953 23275 log.cc:824] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:34.862730 23275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142: No bootstrap required, opened a new log
I20260620 23:12:34.863122 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:34.861262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52125 (local address 127.1.32.126:46727)
0620 23:12:34.861440 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:34.861447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:34.861471 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:12:34.861936 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:34.861943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:34.861968 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:34.862132 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:34.862697 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:34.862700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:34.862700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:34.862704 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:34.862741 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:34.862751 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:34.862857 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:34.862960 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 23:12:34.862994 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260620 23:12:34.864130 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:34.865108 23275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b77de12ef7b478e89af4bbfc1139142" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46727 } }
I20260620 23:12:34.865257 23275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:12:34.865299 23275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b77de12ef7b478e89af4bbfc1139142, State: Initialized, Role: FOLLOWER
I20260620 23:12:34.865401 23275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b77de12ef7b478e89af4bbfc1139142" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46727 } }
I20260620 23:12:34.865473 23275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:12:34.865538 23275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:12:34.865588 23275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:34.866345 23275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b77de12ef7b478e89af4bbfc1139142" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46727 } }
I20260620 23:12:34.866464 23275 leader_election.cc:302] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b77de12ef7b478e89af4bbfc1139142; no voters:
I20260620 23:12:34.866564 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:34.865598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42307 (local address 127.1.32.126:46727)
0620 23:12:34.865630 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:34.865631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:34.865666 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:12:34.865730 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:34.865731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:34.865734 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:34.865774 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:34.866297 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:34.866298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:34.866298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:12:34.866302 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:34.866313 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:12:34.866316 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:34.866354 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:34.866412 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 23:12:34.866453 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:12:34.866637 23275 leader_election.cc:288] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:12:34.866806 23286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:34.866948 23275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:34.867004 23286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [term 1 LEADER]: Becoming Leader. State: Replica: 5b77de12ef7b478e89af4bbfc1139142, State: Running, Role: LEADER
I20260620 23:12:34.867133 23286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b77de12ef7b478e89af4bbfc1139142" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46727 } }
I20260620 23:12:34.867507 23285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b77de12ef7b478e89af4bbfc1139142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b77de12ef7b478e89af4bbfc1139142" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46727 } } }
I20260620 23:12:34.867507 23287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b77de12ef7b478e89af4bbfc1139142. Latest consensus state: current_term: 1 leader_uuid: "5b77de12ef7b478e89af4bbfc1139142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b77de12ef7b478e89af4bbfc1139142" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 46727 } } }
I20260620 23:12:34.867692 23287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:34.868098 23294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:34.868620 23294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:34.867679 23285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:34.870115 23294 catalog_manager.cc:1347] Generated new cluster ID: 5e6a7feda3d34b33b42f03864e44ebbb
I20260620 23:12:34.870195 23294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:34.878353 23294 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:34.878490 23294 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:34.888641 23294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b77de12ef7b478e89af4bbfc1139142: Generated new TSK 0
I20260620 23:12:34.888809 23294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:12:34.896286 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23207
2026-06-20T23:12:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1280 ms)
[----------] 18 tests from AuthTokenIssuingTest (22976 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:34 dist-test-slave-fzjm krb5kdc[23306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:34 dist-test-slave-fzjm krb5kdc[23306](info): set up 2 sockets
Jun 20 23:12:34 dist-test-slave-fzjm krb5kdc[23306](info): commencing operation
krb5kdc: starting...
W20260620 23:12:35.919154 1153 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:35 dist-test-slave-fzjm krb5kdc[23306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:36.029785 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40677
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:40677,127.1.32.125:37587,127.1.32.124:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:36.110106 23319 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:12:36.110316 23319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:36.110332 23319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:36.110349 23319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:36.111436 23319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:36.111474 23319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:36.111487 23319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:36.111500 23319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:36.111511 23319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:36.112854 23319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:40677,127.1.32.125:37587,127.1.32.124:35325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:40677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:36.113029 23319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:36.113178 23319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:12:36.117892 23319 server_base.cc:1034] running on GCE node
W20260620 23:12:36.117882 23325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:36.117872 23324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:36.117872 23327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:36.118296 23319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:36.118583 23319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:36.119750 23319 hybrid_clock.cc:648] HybridClock initialized: now 1781997156119746 us; error 36 us; skew 500 ppm
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:36.122189 23319 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:36.122658 23319 webserver.cc:460] Webserver started at http://127.1.32.126:41653/ using document root <none> and password file <none>
I20260620 23:12:36.122875 23319 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:36.122968 23319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:36.123092 23319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:36.124104 23319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "d1084fa4fd334a49a451e199025bfef7"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "bc2e6bd2061aa88d3031e2a49525930a"
server_key_iv: "e2cddc0ec7d0f05803b1d5774e6587ad"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.124450 23319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "d1084fa4fd334a49a451e199025bfef7"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "bc2e6bd2061aa88d3031e2a49525930a"
server_key_iv: "e2cddc0ec7d0f05803b1d5774e6587ad"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.125777 23319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:36.126493 23334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:36.126703 23319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 23:12:36.126796 23319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "d1084fa4fd334a49a451e199025bfef7"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "bc2e6bd2061aa88d3031e2a49525930a"
server_key_iv: "e2cddc0ec7d0f05803b1d5774e6587ad"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.126848 23319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:36.155275 23319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:36.157680 23319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:36.157796 23319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:36.161798 23319 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:40677
I20260620 23:12:36.161868 23386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:40677 every 8 connection(s)
I20260620 23:12:36.162220 23319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:36.163282 23387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:36.164705 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23319
I20260620 23:12:36.164791 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:36.164978 1153 external_mini_cluster.cc:1428] Setting key 960441f82c3082a71a1bc88ebf0fb920
I20260620 23:12:36.164911 23387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.32.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.125@KRBTEST.COM" created.
I20260620 23:12:36.175894 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.165728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40677 (local address 127.0.0.1:42114)
0620 23:12:36.165948 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.166002 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.166675 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.167047 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.167050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.167416 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.167624 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.167631 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.168661 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.168662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.169864 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.169867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.170532 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.170533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.170616 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.171178 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.171188 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.172634 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.174402 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.174409 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.174411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.174631 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.174872 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.174873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.174873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.174985 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.175345 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.175349 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.175412 (+ 63us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.175504 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.175728 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":96,"thread_start_us":96,"threads_started":1}
I20260620 23:12:36.175897 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.165729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42114 (local address 127.1.32.126:40677)
0620 23:12:36.165932 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.165941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.166102 (+ 161us) server_negotiation.cc:407] Connection header received
0620 23:12:36.166846 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.166849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.166871 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.167012 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.167722 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.168562 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.169953 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.170416 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.170505 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.172737 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.172744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.172744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.172757 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.174341 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.174702 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.174705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.174705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.174776 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.175053 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.175054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.175054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.175275 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.175363 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.175536 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.175663 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260620 23:12:36.176806 23387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:36.177176 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.177047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36136)
0620 23:12:36.177111 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.177157 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 23:12:36.177304 23337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
I20260620 23:12:36.177788 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.177697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36144)
0620 23:12:36.177743 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.177768 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:12:36.177906 23387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:37587: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab.
Entry for principal kudu/127.1.32.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab.
I20260620 23:12:36.189465 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:37587
--webserver_interface=127.1.32.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab
--principal=kudu/127.1.32.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:40677,127.1.32.125:37587,127.1.32.124:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:12:36.227145 23387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } attempt: 1
I20260620 23:12:36.227674 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.227536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36150)
0620 23:12:36.227599 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.227641 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:12:36.228052 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.227971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36160)
0620 23:12:36.228011 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.228037 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:12:36.228173 23387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:37587: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
W20260620 23:12:36.269105 23394 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:12:36.269337 23394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:36.269367 23394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:36.269408 23394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:36.270520 23394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:36.270583 23394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:36.270607 23394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:36.270627 23394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:36.270656 23394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:36.272156 23394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:40677,127.1.32.125:37587,127.1.32.124:35325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab
--principal=kudu/127.1.32.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:37587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.32.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:36.272418 23394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:36.272632 23394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:36.277503 23400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:36.277570 23394 server_base.cc:1034] running on GCE node
W20260620 23:12:36.277472 23399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:36.277472 23402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:36.277961 23394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:36.278237 23394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:36.279373 23394 hybrid_clock.cc:648] HybridClock initialized: now 1781997156279364 us; error 26 us; skew 500 ppm
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:36.281822 23394 init.cc:377] Logged in from keytab as kudu/127.1.32.125@KRBTEST.COM (short username kudu)
I20260620 23:12:36.282271 23394 webserver.cc:460] Webserver started at http://127.1.32.125:40499/ using document root <none> and password file <none>
I20260620 23:12:36.282521 23394 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:36.282589 23394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:36.282738 23394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:36.284191 23394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data/instance:
uuid: "d9ca8ad8efe744b9b70664ac4a27853e"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "cfa30e4e65cf0bceaa2f30e9e7c205ba"
server_key_iv: "41eff49261b5ecf403039b7066c89213"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.284528 23394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/wal/instance:
uuid: "d9ca8ad8efe744b9b70664ac4a27853e"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "cfa30e4e65cf0bceaa2f30e9e7c205ba"
server_key_iv: "41eff49261b5ecf403039b7066c89213"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.285760 23394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 23:12:36.286469 23409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:36.286643 23394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:36.286754 23394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/wal
uuid: "d9ca8ad8efe744b9b70664ac4a27853e"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "cfa30e4e65cf0bceaa2f30e9e7c205ba"
server_key_iv: "41eff49261b5ecf403039b7066c89213"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.286836 23394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:36.295892 23394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:36.296329 23387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } attempt: 2
I20260620 23:12:36.296733 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.296612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36162)
0620 23:12:36.296668 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.296706 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 23:12:36.297080 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.296978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36166)
0620 23:12:36.297022 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.297050 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:12:36.297233 23387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:37587: Network error: Client connection negotiation failed: client connection to 127.1.32.125:37587: connect: Connection refused (error 111)
I20260620 23:12:36.298220 23394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:36.298383 23394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:36.302058 23394 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.125:37587
I20260620 23:12:36.302122 23461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.125:37587 every 8 connection(s)
I20260620 23:12:36.302470 23394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
I20260620 23:12:36.303445 23462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:36.303691 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23394
I20260620 23:12:36.303800 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-1/wal/instance
I20260620 23:12:36.304005 1153 external_mini_cluster.cc:1428] Setting key e58924644fe521e480051ac3cde82f90
I20260620 23:12:36.305047 23462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.32.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.124@KRBTEST.COM" created.
I20260620 23:12:36.314846 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.305792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40677 (local address 127.0.0.1:42130)
0620 23:12:36.305951 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.305990 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.306696 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.306952 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.306956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.307333 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.307609 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.307617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.308247 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.308248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.309522 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.309525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.309899 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.309901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.310062 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.310539 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.310549 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.312090 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.313462 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.313468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.313470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.313681 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.313870 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.313871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.313871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.313977 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.314273 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.314280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.314445 (+ 165us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.314560 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.314695 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260620 23:12:36.314845 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.305825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.1.32.126:40677)
0620 23:12:36.305873 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.305875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.306079 (+ 204us) server_negotiation.cc:407] Connection header received
0620 23:12:36.306794 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.306795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.306807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.306856 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.307701 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.308148 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.309614 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.309794 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.309844 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.312170 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.312171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.312172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.312178 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.313395 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.313746 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.313747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.313747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.313791 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.314022 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.314022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.314023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.314172 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.314254 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.314605 (+ 351us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.314714 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:12:36.315428 23462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:12:36.322454 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.315689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36172)
0620 23:12:36.315745 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.315755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.315791 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.316005 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.316006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.316131 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.316174 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.316175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.316993 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.316994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.317479 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.317481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.317939 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.317940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.318001 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.318380 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.318385 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.319591 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.321363 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.321364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.321365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.321527 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.321774 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.321775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.321776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.321828 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.322116 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.322116 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.322152 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.322220 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.322318 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 23:12:36.322515 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.315740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.1.32.125:37587)
0620 23:12:36.315886 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.315890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.315902 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:12:36.315916 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.315917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.315926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.316004 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.316267 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.316895 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.317568 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.317855 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.317941 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.319688 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.319696 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.319696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.319703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.321253 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.321618 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.321621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.321621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.321699 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.321909 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.321910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.321911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.322047 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.322137 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.322247 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.322366 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260620 23:12:36.323240 23462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:36.323582 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.323465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43452)
0620 23:12:36.323513 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.323555 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 23:12:36.323769 23413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
I20260620 23:12:36.324172 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.324107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43460)
0620 23:12:36.324128 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.324151 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260620 23:12:36.324294 23462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:35325: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab.
Entry for principal kudu/127.1.32.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab.
I20260620 23:12:36.330122 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:35325
--webserver_interface=127.1.32.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab
--principal=kudu/127.1.32.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:40677,127.1.32.125:37587,127.1.32.124:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:12:36.373546 23462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } attempt: 1
I20260620 23:12:36.374140 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.373996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43474)
0620 23:12:36.374063 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.374108 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 23:12:36.374526 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.374420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43486)
0620 23:12:36.374468 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.374503 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 23:12:36.374629 23462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:35325: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
I20260620 23:12:36.388397 23387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } attempt: 3
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:12:36.396598 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.388745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36186)
0620 23:12:36.388789 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.388803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.388854 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.389025 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.389026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.389181 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.389247 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.389265 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.390271 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.390271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.390842 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.390844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.391380 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.391381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.391454 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.391957 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.391962 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.393476 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.395585 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.395586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.395587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.395789 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.396008 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.396009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.396009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.396066 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.396345 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.396346 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.396390 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.396446 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.396530 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 23:12:36.396688 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.388761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36186 (local address 127.1.32.125:37587)
0620 23:12:36.388806 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.388808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.388843 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:12:36.388914 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.388915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.388925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.388999 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.389353 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.390182 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.390943 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.391278 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.391356 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.393565 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.393567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.393568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.393576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.395498 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.395867 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.395869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.395869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.395938 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.396125 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.396126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.396126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.396268 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.396342 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.396488 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.396574 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 23:12:36.397294 23387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:36.397693 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.397578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43500)
0620 23:12:36.397628 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.397672 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 23:12:36.398020 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.397957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43508)
0620 23:12:36.397983 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.398006 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260620 23:12:36.398144 23387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:35325: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
W20260620 23:12:36.412784 23469 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:12:36.412989 23469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:36.413007 23469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:36.413025 23469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:36.414098 23469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:36.414132 23469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:36.414146 23469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:36.414158 23469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:36.414170 23469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:36.415629 23469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:40677,127.1.32.125:37587,127.1.32.124:35325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab
--principal=kudu/127.1.32.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:35325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.32.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:36.415820 23469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:36.415984 23469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:36.420552 23475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:36.420539 23474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:36.420528 23477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:36.420909 23469 server_base.cc:1034] running on GCE node
I20260620 23:12:36.421147 23469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:36.421468 23469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:36.422653 23469 hybrid_clock.cc:648] HybridClock initialized: now 1781997156422655 us; error 44 us; skew 500 ppm
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:36.425194 23469 init.cc:377] Logged in from keytab as kudu/127.1.32.124@KRBTEST.COM (short username kudu)
I20260620 23:12:36.425647 23469 webserver.cc:460] Webserver started at http://127.1.32.124:36245/ using document root <none> and password file <none>
I20260620 23:12:36.425835 23469 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:36.425875 23469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:36.425963 23469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:36.427019 23469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data/instance:
uuid: "5b2f4a6cad814a1baa11b92380ef5c8e"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "e538082c2d38db8a932f2e83d262c1d1"
server_key_iv: "ec426055d8dc2bc460889587c707b8d2"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.427325 23469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/wal/instance:
uuid: "5b2f4a6cad814a1baa11b92380ef5c8e"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "e538082c2d38db8a932f2e83d262c1d1"
server_key_iv: "ec426055d8dc2bc460889587c707b8d2"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.428714 23469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:36.429286 23387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } attempt: 1
I20260620 23:12:36.429534 23484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:36.429683 23469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:36.429757 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.429615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43514)
0620 23:12:36.429676 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.429707 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 23:12:36.429790 23469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/wal
uuid: "5b2f4a6cad814a1baa11b92380ef5c8e"
format_stamp: "Formatted at 2026-06-20 23:12:36 on dist-test-slave-fzjm"
server_key: "e538082c2d38db8a932f2e83d262c1d1"
server_key_iv: "ec426055d8dc2bc460889587c707b8d2"
server_key_version: "encryptionkey@0"
I20260620 23:12:36.429867 23469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:36.430122 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.430050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43516)
0620 23:12:36.430077 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.430103 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260620 23:12:36.430245 23387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:35325: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
I20260620 23:12:36.439770 23469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:36.441527 23469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:36.441630 23469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:36.442782 23462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } attempt: 2
I20260620 23:12:36.443310 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.443144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43524)
0620 23:12:36.443222 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.443264 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 23:12:36.443722 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.443605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43532)
0620 23:12:36.443656 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.443692 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 23:12:36.443816 23462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:35325: Network error: Client connection negotiation failed: client connection to 127.1.32.124:35325: connect: Connection refused (error 111)
I20260620 23:12:36.445770 23469 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.124:35325
I20260620 23:12:36.445798 23536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.124:35325 every 8 connection(s)
I20260620 23:12:36.446139 23469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
I20260620 23:12:36.447122 23537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:36.448596 23537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:36.455116 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23469
I20260620 23:12:36.455231 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996897900784-1153-0/minicluster-data/master-2/wal/instance
I20260620 23:12:36.455435 1153 external_mini_cluster.cc:1428] Setting key cf1222060712f1a0b90504a9f848ebfb
I20260620 23:12:36.457602 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.449336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42142 (local address 127.1.32.126:40677)
0620 23:12:36.449425 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.449427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.449622 (+ 195us) server_negotiation.cc:407] Connection header received
0620 23:12:36.450097 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.450098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.450109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.450178 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.450775 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.451313 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.452269 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.452457 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.452522 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.454707 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.454709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.454709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.454716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.456247 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.456655 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.456657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.456657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.456724 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.456980 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.456981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.456981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.457096 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.457166 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.457435 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.457515 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 23:12:36.457798 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.449307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:40677 (local address 127.0.0.1:42142)
0620 23:12:36.449487 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.449527 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.450009 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.450252 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.450255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.450494 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.450695 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.450699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.451394 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.451396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.452171 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.452173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.452558 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.452559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.452683 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.453210 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.453219 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.454609 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.456339 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.456346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.456348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.456565 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.456790 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.456791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.456791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.456906 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.457159 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.457165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.457306 (+ 141us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.457413 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.457561 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260620 23:12:36.458348 23537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:12:36.464327 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.455790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.1.32.124:35325)
0620 23:12:36.455961 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.455966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.455989 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:12:36.456061 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.456063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.456076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.456144 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.456703 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.457342 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.458323 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.458696 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.458756 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.461296 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.461303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.461303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.461310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.463115 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.463534 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.463537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.463537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.463596 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.463799 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.463799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.463800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.463969 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.464109 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.464153 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.464232 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260620 23:12:36.465332 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.458592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:37587 (local address 127.0.0.1:36188)
0620 23:12:36.458638 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.458649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.458688 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.458848 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.458848 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.459004 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.459047 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.459048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.459713 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.459714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.460414 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.460416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.460886 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.460887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.460952 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.461272 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.461278 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.462338 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.464196 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.464196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.464197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.464416 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.464671 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.464672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.464672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.464730 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.465004 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.465005 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.465047 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.465112 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.465234 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 23:12:36.465339 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.458655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36188 (local address 127.1.32.125:37587)
0620 23:12:36.458703 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.458705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.458712 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:36.458739 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.458740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.458750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.458837 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.459106 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.459637 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.460494 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.460825 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.460886 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.462429 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.462431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.462431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.462438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.464119 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.464517 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.464518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.464519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.464596 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.464795 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.464796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.464796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.464935 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.465001 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.465158 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.465240 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:12:36.466032 23537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:12:36.472559 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.465240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.1.32.126:40677)
0620 23:12:36.465276 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.465278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.465285 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:36.465304 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.465305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.465313 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.465368 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.465848 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.466524 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.467443 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.467719 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.467789 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.469930 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.469932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.469932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.469939 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.471429 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.471773 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.471774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.471774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.471819 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.472026 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.472027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.472027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.472153 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.472342 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.472380 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.472451 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:36.473078 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.466283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43548)
0620 23:12:36.466333 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.466344 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.466384 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.466488 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.466489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.466636 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.466671 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.466673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.467506 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.467506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.468222 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.468225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.468759 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.468760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.468795 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.469171 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.469175 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.470306 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.472170 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.472171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.472171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.472366 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.472552 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.472553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.472553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.472603 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.472830 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.472831 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.472877 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.472946 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.473024 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:12:36.473187 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.466329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.1.32.124:35325)
0620 23:12:36.466358 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.466359 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.466371 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:12:36.466439 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.466439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.466446 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.466476 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.466740 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.467443 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.468293 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.468622 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.468750 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.470379 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.470380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.470380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.470386 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.472091 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.472440 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.472441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.472441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.472493 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.472669 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.472670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.472670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.472779 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.472833 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.472986 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.473077 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":13}
I20260620 23:12:36.474589 23537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: Bootstrap starting.
I20260620 23:12:36.475548 23537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:36.475874 23537 log.cc:824] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:36.476748 23537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: No bootstrap required, opened a new log
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:12:36.478349 23537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.478488 23537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2f4a6cad814a1baa11b92380ef5c8e, State: Initialized, Role: FOLLOWER
I20260620 23:12:36.478575 23537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.478928 23537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:36.479044 23543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.479161 23543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:12:36.480623 23554 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:36.480749 23554 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:36.480926 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.473196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.1.32.125:37587)
0620 23:12:36.473233 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.473235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.473332 (+ 97us) server_negotiation.cc:407] Connection header received
0620 23:12:36.473365 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.473366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.473375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.473424 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.474006 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.474433 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.475375 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.475562 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.475644 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.477999 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.478000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.478000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.478007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.479584 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.479934 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.479935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.479935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.479986 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.480186 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.480187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.480187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.480331 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.480476 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.480514 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.480579 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:36.482069 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:36.480960 23469 master_runner.cc:186] Error getting master registration for 127.1.32.126:40677: OK, instance_id { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" instance_seqno: 1781997156160589 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:12:36.483227 23469 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:12:36.483296 23361 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d1084fa4fd334a49a451e199025bfef7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42152
I20260620 23:12:36.505398 23387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } attempt: 2
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:12:36.513129 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.505768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43552)
0620 23:12:36.505829 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.505840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.505890 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.506066 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.506067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.506270 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.506323 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.506325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.507242 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.507243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.508035 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.508037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.508504 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.508505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.508577 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.509040 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.509045 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.510196 (+ 1151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.512029 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.512030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.512030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.512207 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.512421 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.512422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.512422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.512480 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.512807 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.512808 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.512856 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.512924 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.513020 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:12:36.513130 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.505799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.1.32.124:35325)
0620 23:12:36.505859 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.505861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.505873 (+ 12us) server_negotiation.cc:407] Connection header received
0620 23:12:36.505968 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.505969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.505980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.506047 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.506407 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.507168 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.508124 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.508424 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.508494 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.510278 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.510279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.510279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.510286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.511928 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.512276 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.512278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.512278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.512338 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.512551 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.512551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.512552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.512737 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.512808 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.512962 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.513040 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:36.515198 23387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7: Bootstrap starting.
I20260620 23:12:36.515925 23387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:36.516207 23387 log.cc:824] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:36.516992 23387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7: No bootstrap required, opened a new log
I20260620 23:12:36.518353 23387 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.518503 23387 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1084fa4fd334a49a451e199025bfef7, State: Initialized, Role: FOLLOWER
I20260620 23:12:36.518571 23387 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.518862 23387 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:36.518874 23361 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 23:12:36.518973 23361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:36.519802 23361 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.520062 23361 leader_election.cc:288] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [CANDIDATE]: Term 1 election: Requested vote from peers d9ca8ad8efe744b9b70664ac4a27853e (127.1.32.125:37587), 5b2f4a6cad814a1baa11b92380ef5c8e (127.1.32.124:35325)
I20260620 23:12:36.520380 23511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1084fa4fd334a49a451e199025bfef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e"
I20260620 23:12:36.520520 23511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:36.521375 23511 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1084fa4fd334a49a451e199025bfef7 in term 1.
I20260620 23:12:36.521708 23338 leader_election.cc:302] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2f4a6cad814a1baa11b92380ef5c8e, d1084fa4fd334a49a451e199025bfef7; no voters:
I20260620 23:12:36.521920 23573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:36.520352 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.522061 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:36.522203 23573 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [term 1 LEADER]: Becoming Leader. State: Replica: d1084fa4fd334a49a451e199025bfef7, State: Running, Role: LEADER
W20260620 23:12:36.522547 23568 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:36.522650 23568 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:36.522324 23573 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.523671 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1084fa4fd334a49a451e199025bfef7. Latest consensus state: current_term: 1 leader_uuid: "d1084fa4fd334a49a451e199025bfef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.523761 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:36.524811 23319 master_runner.cc:186] Error getting master registration for 127.1.32.125:37587: OK, instance_id { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" instance_seqno: 1781997156301108 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:12:36.527235 23319 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:12:36.531113 23580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:36.531353 23580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:36.532963 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.521580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.1.32.126:40677)
0620 23:12:36.521645 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.521647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.524398 (+ 2751us) server_negotiation.cc:407] Connection header received
0620 23:12:36.524493 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.524494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.524504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.524534 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.525260 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.526004 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.527214 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.527495 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.527533 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.529165 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.529166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.529167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.529173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.531561 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.532084 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.532085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.532085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.532144 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.532352 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.532353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.532353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.532486 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.532563 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.532822 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.532886 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:12:36.533283 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.522508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.1.32.125:37587)
0620 23:12:36.522552 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.522553 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.523375 (+ 822us) server_negotiation.cc:407] Connection header received
0620 23:12:36.523487 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.523488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.523500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.523532 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.525288 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.526020 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.527179 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.527896 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.527965 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.529748 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.529749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.529750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.529757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.531632 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.532252 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.532253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.532253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.532320 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.532601 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.532602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.532602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.532752 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.532830 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.533110 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.533201 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:36.534929 23462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } attempt: 3
I20260620 23:12:36.539422 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.523013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43560 (local address 127.1.32.124:35325)
0620 23:12:36.523142 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.523144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.531018 (+ 7874us) server_negotiation.cc:407] Connection header received
0620 23:12:36.532150 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.532152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.532163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.532204 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.532795 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.533439 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.534313 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.534619 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.534687 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.536240 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.536241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.536242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.536249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.537913 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.538293 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.538294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.538294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.538348 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.538540 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.538541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.538541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.538661 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.538741 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.539246 (+ 505us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.539321 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
Jun 20 23:12:36 dist-test-slave-fzjm krb5kdc[23306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:12:36.543748 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.535182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:35325 (local address 127.0.0.1:43570)
0620 23:12:36.535218 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:36.535229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:36.535267 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:36.535625 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:36.535626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:36.535793 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:36.535830 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.535832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.537194 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.537195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.537992 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:36.537994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.538690 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.538691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:36.538757 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.539150 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:36.539154 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:36.540581 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:36.542625 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.542626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.542627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.542839 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.543124 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:36.543125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:36.543125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:36.543180 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:36.543457 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:36.543458 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:36.543499 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 23:12:36.543563 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 23:12:36.543648 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260620 23:12:36.543797 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:36.535271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.1.32.124:35325)
0620 23:12:36.535454 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.535456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.535463 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:36.535482 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.535482 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.535514 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.535904 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.535956 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.537068 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.538097 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:36.538581 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:36.538924 (+ 343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:36.540701 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:36.540702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:36.540702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:36.540708 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:36.542534 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.542988 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.542989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.542989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.543049 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:36.543244 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:36.543244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:36.543245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:36.543384 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:36.543463 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:36.543600 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 23:12:36.543684 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260620 23:12:36.545650 23462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: Bootstrap starting.
I20260620 23:12:36.546380 23462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:36.546659 23462 log.cc:824] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:36.547886 23511 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [term 1 FOLLOWER]: Refusing update from remote peer d1084fa4fd334a49a451e199025bfef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:36.548229 23462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: No bootstrap required, opened a new log
I20260620 23:12:36.548218 23574 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:36.549468 23543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1084fa4fd334a49a451e199025bfef7. Latest consensus state: current_term: 1 leader_uuid: "d1084fa4fd334a49a451e199025bfef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.549551 23543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:36.550411 23573 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1084fa4fd334a49a451e199025bfef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.550526 23573 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:36.550513 23543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1084fa4fd334a49a451e199025bfef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.550592 23543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:36.550596 23581 mvcc.cc:205] Tried to move back new op lower bound from 7299060353217884160 to 7299060353119313920. Current Snapshot: MvccSnapshot[applied={T|T < 7299060353217884160}]
I20260620 23:12:36.551215 23584 mvcc.cc:205] Tried to move back new op lower bound from 7299060353217884160 to 7299060353119313920. Current Snapshot: MvccSnapshot[applied={T|T < 7299060353217884160}]
I20260620 23:12:36.551915 23462 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.552167 23462 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ca8ad8efe744b9b70664ac4a27853e, State: Initialized, Role: FOLLOWER
I20260620 23:12:36.552327 23462 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } }
I20260620 23:12:36.552860 23462 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:36.552906 23435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:36.552913 23436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1084fa4fd334a49a451e199025bfef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d9ca8ad8efe744b9b70664ac4a27853e"
I20260620 23:12:36.552806 23588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.553224 23588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:36.553790 23435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [term 1 FOLLOWER]: Refusing update from remote peer d1084fa4fd334a49a451e199025bfef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:36.554067 23574 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:36.555056 23436 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1084fa4fd334a49a451e199025bfef7 in term 1.
I20260620 23:12:36.555336 23580 catalog_manager.cc:1347] Generated new cluster ID: deb9a097cdb946e298a3a0ee11adf743
I20260620 23:12:36.555377 23580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:12:36.559674 23588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1084fa4fd334a49a451e199025bfef7. Latest consensus state: current_term: 1 leader_uuid: "d1084fa4fd334a49a451e199025bfef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.559837 23588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:36.559909 23588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1084fa4fd334a49a451e199025bfef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.559996 23588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:12:36.560501 23602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:36.560591 23602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:36.561875 23589 mvcc.cc:205] Tried to move back new op lower bound from 7299060353217884160 to 7299060353119313920. Current Snapshot: MvccSnapshot[applied={T|T < 7299060353217884160}]
I20260620 23:12:36.577745 23573 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1084fa4fd334a49a451e199025bfef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1084fa4fd334a49a451e199025bfef7" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 40677 } } peers { permanent_uuid: "d9ca8ad8efe744b9b70664ac4a27853e" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 37587 } } peers { permanent_uuid: "5b2f4a6cad814a1baa11b92380ef5c8e" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 35325 } } }
I20260620 23:12:36.580024 23573 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:36.580189 23580 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:36.581380 23580 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:36.591041 23580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1084fa4fd334a49a451e199025bfef7: Generated new TSK 0
I20260620 23:12:36.591251 23580 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:12:36.618418 23389 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.616770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.1.32.126:40677)
0620 23:12:36.616814 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.616816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.617738 (+ 922us) server_negotiation.cc:407] Connection header received
0620 23:12:36.617857 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.617859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.617870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.617902 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.618348 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42168: BlockingRecv error: recv got EOF from 127.0.0.1:42168 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 23:12:36.619370 23582 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.617127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43584 (local address 127.1.32.124:35325)
0620 23:12:36.617168 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.617170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.618628 (+ 1458us) server_negotiation.cc:407] Connection header received
0620 23:12:36.618853 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.618854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.618864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.618921 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.619314 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43584: BlockingRecv error: recv got EOF from 127.0.0.1:43584 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 23:12:36.619913 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.617453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36210 (local address 127.1.32.125:37587)
0620 23:12:36.617493 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.617495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.618804 (+ 1309us) server_negotiation.cc:407] Connection header received
0620 23:12:36.619513 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.619514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.619525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.619553 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.619856 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36210: BlockingRecv error: recv got EOF from 127.0.0.1:36210 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 23:12:36.621883 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.621130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.1.32.125:37587)
0620 23:12:36.621163 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.621164 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.621402 (+ 238us) server_negotiation.cc:407] Connection header received
0620 23:12:36.621507 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.621508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.621526 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.621555 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.621843 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36216: BlockingRecv error: recv got EOF from 127.0.0.1:36216 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 23:12:36.624766 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.624078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.1.32.125:37587)
0620 23:12:36.624110 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.624112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.624322 (+ 210us) server_negotiation.cc:407] Connection header received
0620 23:12:36.624412 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.624412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.624421 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.624449 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.624728 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36230: BlockingRecv error: recv got EOF from 127.0.0.1:36230 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:12:36.628479 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.627777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.1.32.125:37587)
0620 23:12:36.627808 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.627809 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.628030 (+ 221us) server_negotiation.cc:407] Connection header received
0620 23:12:36.628116 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.628117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.628126 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.628152 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.628438 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36234: BlockingRecv error: recv got EOF from 127.0.0.1:36234 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:12:36.634308 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.633455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36240 (local address 127.1.32.125:37587)
0620 23:12:36.633484 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.633486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.633774 (+ 288us) server_negotiation.cc:407] Connection header received
0620 23:12:36.633896 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.633897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.633906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.633935 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.634256 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36240: BlockingRecv error: recv got EOF from 127.0.0.1:36240 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:12:36.644388 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.643511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.1.32.125:37587)
0620 23:12:36.643559 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.643561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.643820 (+ 259us) server_negotiation.cc:407] Connection header received
0620 23:12:36.643936 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.643937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.643948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.643977 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.644335 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36242: BlockingRecv error: recv got EOF from 127.0.0.1:36242 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:12:36.662381 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.661547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.1.32.125:37587)
0620 23:12:36.661588 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.661590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.661833 (+ 243us) server_negotiation.cc:407] Connection header received
0620 23:12:36.661928 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.661929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.661939 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.661979 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.662322 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36246: BlockingRecv error: recv got EOF from 127.0.0.1:36246 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:12:36.696435 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.695743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.1.32.125:37587)
0620 23:12:36.695789 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.695791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.695919 (+ 128us) server_negotiation.cc:407] Connection header received
0620 23:12:36.695987 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.695988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.695999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.696064 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.696370 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36252: BlockingRecv error: recv got EOF from 127.0.0.1:36252 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:12:36.762241 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.761538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.1.32.125:37587)
0620 23:12:36.761631 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.761633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.761719 (+ 86us) server_negotiation.cc:407] Connection header received
0620 23:12:36.761744 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.761745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.761756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.761796 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.762151 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36268: BlockingRecv error: recv got EOF from 127.0.0.1:36268 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260620 23:12:36.892109 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:36.891420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36280 (local address 127.1.32.125:37587)
0620 23:12:36.891534 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:36.891536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:36.891560 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:12:36.891638 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:36.891638 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:36.891648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:36.891717 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:36.892053 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36280: BlockingRecv error: recv got EOF from 127.0.0.1:36280 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260620 23:12:37.150051 23467 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:37.149356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36294 (local address 127.1.32.125:37587)
0620 23:12:37.149453 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:37.149455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:37.149465 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:12:37.149506 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:37.149507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:37.149525 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:37.149594 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:37.150002 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36294: BlockingRecv error: recv got EOF from 127.0.0.1:36294 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 23:12:37.481280 23554 catalog_manager.cc:1259] Loaded cluster ID: deb9a097cdb946e298a3a0ee11adf743
I20260620 23:12:37.481374 23554 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: loading cluster ID for follower catalog manager: success
I20260620 23:12:37.482499 23554 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: acquiring CA information for follower catalog manager: success
I20260620 23:12:37.482833 23554 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b2f4a6cad814a1baa11b92380ef5c8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:12:37.561015 23602 catalog_manager.cc:1259] Loaded cluster ID: deb9a097cdb946e298a3a0ee11adf743
I20260620 23:12:37.561084 23602 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: loading cluster ID for follower catalog manager: success
I20260620 23:12:37.561833 23602 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: acquiring CA information for follower catalog manager: success
I20260620 23:12:37.562088 23602 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d9ca8ad8efe744b9b70664ac4a27853e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:12:37.665726 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:37.663299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.1.32.125:37587)
0620 23:12:37.663417 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:37.663418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:37.663428 (+ 10us) server_negotiation.cc:407] Connection header received
0620 23:12:37.663487 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:37.663487 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:37.663535 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:37.663604 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:37.663763 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.664243 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.664949 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.665123 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.665191 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:37.665386 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:37.665489 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:37.665542 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:37.665582 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 23:12:37.665598 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
I20260620 23:12:37.669734 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:37.667580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43592 (local address 127.1.32.124:35325)
0620 23:12:37.667673 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:37.667675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:37.667683 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:37.667741 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:37.667742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:37.667774 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:37.667844 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:37.667917 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.668369 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.669081 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.669259 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.669305 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:37.669425 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:37.669509 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:37.669560 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:37.669596 (+ 36us) server_negotiation.cc:299] Negotiation successful
0620 23:12:37.669609 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":52,"threads_started":1}
I20260620 23:12:37.674124 23682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:37.671379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42182 (local address 127.1.32.126:40677)
0620 23:12:37.671588 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:37.671590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:37.671598 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:37.671638 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:37.671639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:37.671704 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:37.671778 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:37.671947 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.672483 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.673278 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.673489 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.673573 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:37.673788 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:37.673855 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:37.673929 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:37.674003 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 23:12:37.674024 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":58,"threads_started":1}
I20260620 23:12:37.674350 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:37.671512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.1.32.125:37587)
0620 23:12:37.671562 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:37.671564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:37.671571 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:37.671598 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:37.671598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:37.671608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:37.671665 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:37.671783 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.672489 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.673594 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.673863 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.673913 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:37.674099 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:37.674129 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:37.674182 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:37.674238 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 23:12:37.674258 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 23:12:37.674479 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:37.671567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.1.32.124:35325)
0620 23:12:37.671629 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:37.671631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:37.671912 (+ 281us) server_negotiation.cc:407] Connection header received
0620 23:12:37.671958 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:37.671960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:37.671970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:37.672023 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:37.672109 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.672750 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.673716 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:37.674017 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:37.674055 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:37.674252 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:37.674277 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:37.674304 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:37.674406 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:12:37.674426 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 23:12:37.676158 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23319
I20260620 23:12:37.681816 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23394
I20260620 23:12:37.686619 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23469
2026-06-20T23:12:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2792 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:12:37 dist-test-slave-fzjm krb5kdc[23686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:12:37 dist-test-slave-fzjm krb5kdc[23686](info): set up 2 sockets
Jun 20 23:12:37 dist-test-slave-fzjm krb5kdc[23686](info): commencing operation
krb5kdc: starting...
W20260620 23:12:38.699112 1153 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/test-user.keytab.
Jun 20 23:12:38 dist-test-slave-fzjm krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.32.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
Entry for principal kudu/127.1.32.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab.
I20260620 23:12:38.809864 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:34863
--webserver_interface=127.1.32.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:34863,127.1.32.125:41775,127.1.32.124:34269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:12:38.888684 23699 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:12:38.888880 23699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:38.888898 23699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:38.888917 23699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:38.890123 23699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:38.890169 23699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:38.890183 23699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:38.890196 23699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:38.890228 23699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:38.891685 23699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:34863,127.1.32.125:41775,127.1.32.124:34269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.126.keytab
--principal=kudu/127.1.32.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.126:34863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.32.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:38.891927 23699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:38.892158 23699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:38.897040 23705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:38.897046 23704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:38.897120 23699 server_base.cc:1034] running on GCE node
W20260620 23:12:38.897017 23707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:38.897382 23699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:38.897629 23699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:38.898766 23699 hybrid_clock.cc:648] HybridClock initialized: now 1781997158898757 us; error 26 us; skew 500 ppm
Jun 20 23:12:38 dist-test-slave-fzjm krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:38.901226 23699 init.cc:377] Logged in from keytab as kudu/127.1.32.126@KRBTEST.COM (short username kudu)
I20260620 23:12:38.901692 23699 webserver.cc:460] Webserver started at http://127.1.32.126:35139/ using document root <none> and password file <none>
I20260620 23:12:38.901918 23699 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:38.901996 23699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:38.902158 23699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:38.903228 23699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data/instance:
uuid: "fe5bb98165024583a3486646a1dadf35"
format_stamp: "Formatted at 2026-06-20 23:12:38 on dist-test-slave-fzjm"
server_key: "89372b3639ab45e4e20a3779455d8ce8"
server_key_iv: "a8757bc412bcd36ebf737dcdd1f411d8"
server_key_version: "encryptionkey@0"
I20260620 23:12:38.903559 23699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/wal/instance:
uuid: "fe5bb98165024583a3486646a1dadf35"
format_stamp: "Formatted at 2026-06-20 23:12:38 on dist-test-slave-fzjm"
server_key: "89372b3639ab45e4e20a3779455d8ce8"
server_key_iv: "a8757bc412bcd36ebf737dcdd1f411d8"
server_key_version: "encryptionkey@0"
I20260620 23:12:38.904873 23699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 23:12:38.905611 23714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:38.905805 23699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 23:12:38.905913 23699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/wal
uuid: "fe5bb98165024583a3486646a1dadf35"
format_stamp: "Formatted at 2026-06-20 23:12:38 on dist-test-slave-fzjm"
server_key: "89372b3639ab45e4e20a3779455d8ce8"
server_key_iv: "a8757bc412bcd36ebf737dcdd1f411d8"
server_key_version: "encryptionkey@0"
I20260620 23:12:38.905987 23699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:38.913962 23699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:38.916081 23699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:38.916236 23699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:38.920413 23699 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.126:34863
I20260620 23:12:38.920470 23766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.126:34863 every 8 connection(s)
I20260620 23:12:38.920801 23699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/data/info.pb
I20260620 23:12:38.921803 23767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:38.923483 23767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:38.924048 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23699
I20260620 23:12:38.924126 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-0/wal/instance
I20260620 23:12:38.924346 1153 external_mini_cluster.cc:1428] Setting key a31d011c13816fcec8201d536f77a6c2
Jun 20 23:12:38 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.32.125@KRBTEST.COM; defaulting to no policy
I20260620 23:12:38.933913 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:38.924289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.1.32.126:34863)
0620 23:12:38.924495 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:38.924506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:38.924655 (+ 149us) server_negotiation.cc:407] Connection header received
0620 23:12:38.925163 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:38.925166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:38.925188 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:38.925296 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:38.925899 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:38.926759 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:38.927673 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:38.928206 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:38.928346 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:38.930535 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:38.930543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:38.930543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:38.930551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:38.932463 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:38.932831 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:38.932833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:38.932834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:38.932909 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:38.933212 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:38.933212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:38.933213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:38.933412 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:38.933474 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:38.933655 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 23:12:38.933745 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":100,"threads_started":1}
I20260620 23:12:38.934098 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:38.924278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:34863 (local address 127.0.0.1:34664)
0620 23:12:38.924519 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:38.924556 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:38.925016 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:38.925296 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:38.925298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:38.925535 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:38.925790 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:38.925796 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:38.926850 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:38.926851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:38.927584 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:38.927586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:38.928346 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:38.928347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:38.928406 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:38.929014 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:38.929022 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:38.930446 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:38.932537 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:38.932543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:38.932545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:38.932749 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:38.933014 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:38.933015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:38.933016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:38.933149 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:38.933475 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:38.933480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:38.933536 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 23:12:38.933614 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:12:38.933826 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":77,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.125@KRBTEST.COM" created.
I20260620 23:12:38.934881 23767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:38.935253 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:38.935142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50156)
0620 23:12:38.935191 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:38.935234 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 23:12:38.935398 23716 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
I20260620 23:12:38.935889 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:38.935790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50158)
0620 23:12:38.935839 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:38.935865 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 23:12:38.936007 23767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:41775: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab.
Entry for principal kudu/127.1.32.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab.
I20260620 23:12:38.947613 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:41775
--webserver_interface=127.1.32.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab
--principal=kudu/127.1.32.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:34863,127.1.32.125:41775,127.1.32.124:34269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:12:38.985236 23767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } attempt: 1
I20260620 23:12:38.985714 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:38.985565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50168)
0620 23:12:38.985643 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:38.985686 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:12:38.986068 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:38.985977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50170)
0620 23:12:38.986020 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:38.986044 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 23:12:38.986156 23767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:41775: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
W20260620 23:12:39.026146 23774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:12:39.026423 23774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:39.026455 23774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:39.026499 23774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:39.027661 23774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:39.027745 23774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:39.027769 23774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:39.027791 23774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:39.027819 23774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:39.029237 23774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:34863,127.1.32.125:41775,127.1.32.124:34269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.125.keytab
--principal=kudu/127.1.32.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.125:41775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.32.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:39.029515 23774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:39.029740 23774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:39.034466 23779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:39.034552 23774 server_base.cc:1034] running on GCE node
W20260620 23:12:39.034456 23782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:39.034476 23780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:39.034999 23774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:39.035328 23774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:39.036505 23774 hybrid_clock.cc:648] HybridClock initialized: now 1781997159036490 us; error 28 us; skew 500 ppm
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:39.038973 23774 init.cc:377] Logged in from keytab as kudu/127.1.32.125@KRBTEST.COM (short username kudu)
I20260620 23:12:39.039443 23774 webserver.cc:460] Webserver started at http://127.1.32.125:37997/ using document root <none> and password file <none>
I20260620 23:12:39.039687 23774 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:39.039753 23774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:39.039898 23774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:39.040957 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data/instance:
uuid: "e784f45688ac41f09402180ca3049b3f"
format_stamp: "Formatted at 2026-06-20 23:12:39 on dist-test-slave-fzjm"
server_key: "cd53c98b24c9426249730aca8d960df5"
server_key_iv: "17d6a09961e68d08c223cddac34320d4"
server_key_version: "encryptionkey@0"
I20260620 23:12:39.041375 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/wal/instance:
uuid: "e784f45688ac41f09402180ca3049b3f"
format_stamp: "Formatted at 2026-06-20 23:12:39 on dist-test-slave-fzjm"
server_key: "cd53c98b24c9426249730aca8d960df5"
server_key_iv: "17d6a09961e68d08c223cddac34320d4"
server_key_version: "encryptionkey@0"
I20260620 23:12:39.042698 23774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 23:12:39.043468 23789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:39.043627 23774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:39.043731 23774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/wal
uuid: "e784f45688ac41f09402180ca3049b3f"
format_stamp: "Formatted at 2026-06-20 23:12:39 on dist-test-slave-fzjm"
server_key: "cd53c98b24c9426249730aca8d960df5"
server_key_iv: "17d6a09961e68d08c223cddac34320d4"
server_key_version: "encryptionkey@0"
I20260620 23:12:39.043825 23774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:39.054296 23767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } attempt: 2
I20260620 23:12:39.054540 23774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:39.054687 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.054551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50182)
0620 23:12:39.054608 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.054640 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:12:39.055086 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.054998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50196)
0620 23:12:39.055034 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.055061 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260620 23:12:39.055197 23767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.125:41775: Network error: Client connection negotiation failed: client connection to 127.1.32.125:41775: connect: Connection refused (error 111)
I20260620 23:12:39.056624 23774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:39.056726 23774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:39.060637 23774 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.125:41775
I20260620 23:12:39.060699 23841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.125:41775 every 8 connection(s)
I20260620 23:12:39.061017 23774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/data/info.pb
I20260620 23:12:39.061664 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23774
I20260620 23:12:39.061766 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-1/wal/instance
I20260620 23:12:39.061951 1153 external_mini_cluster.cc:1428] Setting key e779e3a10ee36848635920e0a7bc27df
I20260620 23:12:39.062165 23842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:39.063747 23842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.32.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.32.124@KRBTEST.COM" created.
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:39.075304 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.064475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.1.32.126:34863)
0620 23:12:39.064536 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.064538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.064724 (+ 186us) server_negotiation.cc:407] Connection header received
0620 23:12:39.065238 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.065240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.065255 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.065326 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.065908 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.066429 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.067495 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.067715 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.067798 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.072171 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.072173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.072173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.072182 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.073927 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.074334 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.074336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.074336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.074393 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.074632 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.074633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.074634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.074762 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.074833 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.075128 (+ 295us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.075211 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:12:39.075502 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.064453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:34863 (local address 127.0.0.1:34680)
0620 23:12:39.064609 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.064643 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.065122 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.065381 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.065385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.065624 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.065803 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.065807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.066542 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.066543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.067393 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.067396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.067837 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.067839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.067952 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.068487 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.068494 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.072068 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.074022 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.074029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.074032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.074259 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.074462 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.074463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.074463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.074566 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.074837 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.074844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.075005 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.075111 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.075270 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260620 23:12:39.076027 23842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:12:39.083760 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.076318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50212)
0620 23:12:39.076347 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.076355 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.076387 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.076643 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.076644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.076779 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.076825 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.076826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.077665 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.077666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.078179 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.078180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.078647 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.078648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.078709 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.079098 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.079102 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.080911 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.082556 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.082557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.082557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.082713 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.082997 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.082998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.082998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.083051 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.083356 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.083357 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.083394 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.083464 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.083656 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260620 23:12:39.083868 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.076363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.1.32.125:41775)
0620 23:12:39.076521 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.076525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.076539 (+ 14us) server_negotiation.cc:407] Connection header received
0620 23:12:39.076553 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.076555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.076565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.076643 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.076895 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.077548 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.078263 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.078563 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.078648 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.081006 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.081013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.081014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.081020 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.082469 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.082801 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.082803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.082804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.082857 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.083137 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.083138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.083138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.083287 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.083379 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.083493 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.083743 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":152,"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260620 23:12:39.084558 23842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:39.084874 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.084779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55900)
0620 23:12:39.084816 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.084856 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260620 23:12:39.085021 23791 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
I20260620 23:12:39.085425 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.085369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55912)
0620 23:12:39.085390 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.085411 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.32.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab.
Entry for principal kudu/127.1.32.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab.
W20260620 23:12:39.085608 23842 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:34269: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
I20260620 23:12:39.086246 1153 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTMrFo/build/release/bin/kudu
/tmp/dist-test-taskfTMrFo/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:34269
--webserver_interface=127.1.32.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab
--principal=kudu/127.1.32.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.32.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.32.126:34863,127.1.32.125:41775,127.1.32.124:34269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:12:39.134858 23842 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } attempt: 1
I20260620 23:12:39.135368 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.135239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55920)
0620 23:12:39.135298 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.135338 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 23:12:39.135718 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.135623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55924)
0620 23:12:39.135669 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.135693 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 23:12:39.135805 23842 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:34269: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
I20260620 23:12:39.146339 23767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } attempt: 3
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:12:39.154296 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.146699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50220)
0620 23:12:39.146758 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.146772 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.146822 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.147049 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.147050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.147196 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.147287 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.147290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.148337 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.148338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.148930 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.148932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.149489 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.149489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.149547 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.150070 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.150074 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.151490 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.153299 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.153300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.153300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.153465 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.153670 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.153671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.153671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.153722 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.154015 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.154016 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.154064 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.154134 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.154227 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 23:12:39.154356 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.146786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50220 (local address 127.1.32.125:41775)
0620 23:12:39.146840 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.146842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.146918 (+ 76us) server_negotiation.cc:407] Connection header received
0620 23:12:39.146951 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.146952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.146965 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.147023 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.147352 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.148208 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.149022 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.149409 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.149488 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.151608 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.151609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.151610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.151618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.153215 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.153539 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.153540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.153541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.153603 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.153782 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.153783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.153784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.153924 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.153996 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.154175 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.154255 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 23:12:39.154785 23767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:39.155148 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.155034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55934)
0620 23:12:39.155084 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.155119 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 23:12:39.155498 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.155416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55948)
0620 23:12:39.155460 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.155484 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 23:12:39.155614 23767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:34269: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
W20260620 23:12:39.168035 23849 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:12:39.168239 23849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:12:39.168255 23849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:12:39.168274 23849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:12:39.169337 23849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:12:39.169370 23849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:12:39.169384 23849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:12:39.169396 23849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:12:39.169409 23849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:12:39.170799 23849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.32.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.32.126:34863,127.1.32.125:41775,127.1.32.124:34269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/krb5kdc/kudu_127.1.32.124.keytab
--principal=kudu/127.1.32.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.32.124:34269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.32.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 22:11:48 UTC on ccycloud.quasar-qwxqme.root.comops.site
build id 4457789
I20260620 23:12:39.171053 23849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:12:39.171227 23849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:12:39.175649 23857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:39.175642 23854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:12:39.175653 23855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:12:39.175962 23849 server_base.cc:1034] running on GCE node
I20260620 23:12:39.176194 23849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:12:39.176431 23849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:12:39.177590 23849 hybrid_clock.cc:648] HybridClock initialized: now 1781997159177561 us; error 41 us; skew 500 ppm
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:12:39.179963 23849 init.cc:377] Logged in from keytab as kudu/127.1.32.124@KRBTEST.COM (short username kudu)
I20260620 23:12:39.180372 23849 webserver.cc:460] Webserver started at http://127.1.32.124:35159/ using document root <none> and password file <none>
I20260620 23:12:39.180581 23849 fs_manager.cc:359] Metadata directory not provided
I20260620 23:12:39.180624 23849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:12:39.180760 23849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:12:39.181739 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data/instance:
uuid: "36cb4182a4cb476882ab14fa3dc816c3"
format_stamp: "Formatted at 2026-06-20 23:12:39 on dist-test-slave-fzjm"
server_key: "14046d1d61b063866f2c5d63535e45c0"
server_key_iv: "ad121eaa2b6ae9d349f904b772074c34"
server_key_version: "encryptionkey@0"
I20260620 23:12:39.182096 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/wal/instance:
uuid: "36cb4182a4cb476882ab14fa3dc816c3"
format_stamp: "Formatted at 2026-06-20 23:12:39 on dist-test-slave-fzjm"
server_key: "14046d1d61b063866f2c5d63535e45c0"
server_key_iv: "ad121eaa2b6ae9d349f904b772074c34"
server_key_version: "encryptionkey@0"
I20260620 23:12:39.183463 23849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260620 23:12:39.184193 23864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:12:39.184362 23849 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 23:12:39.184458 23849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/wal
uuid: "36cb4182a4cb476882ab14fa3dc816c3"
format_stamp: "Formatted at 2026-06-20 23:12:39 on dist-test-slave-fzjm"
server_key: "14046d1d61b063866f2c5d63535e45c0"
server_key_iv: "ad121eaa2b6ae9d349f904b772074c34"
server_key_version: "encryptionkey@0"
I20260620 23:12:39.184530 23849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:12:39.186779 23767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } attempt: 1
I20260620 23:12:39.187165 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.187043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55958)
0620 23:12:39.187099 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.187131 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 23:12:39.187460 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.187405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55962)
0620 23:12:39.187427 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.187446 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260620 23:12:39.187561 23767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:34269: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
I20260620 23:12:39.203938 23842 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } attempt: 2
I20260620 23:12:39.204294 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.204175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55974)
0620 23:12:39.204230 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.204262 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 23:12:39.204602 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.204514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:55990)
0620 23:12:39.204556 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.204578 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 23:12:39.204722 23842 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.32.124:34269: Network error: Client connection negotiation failed: client connection to 127.1.32.124:34269: connect: Connection refused (error 111)
I20260620 23:12:39.211519 23849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:12:39.213569 23849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:12:39.213722 23849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:12:39.217552 23849 rpc_server.cc:307] RPC server started. Bound to: 127.1.32.124:34269
I20260620 23:12:39.217583 23916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.32.124:34269 every 8 connection(s)
I20260620 23:12:39.217937 23849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/data/info.pb
I20260620 23:12:39.219004 23917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:12:39.220512 23917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:39.220703 1153 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTMrFo/build/release/bin/kudu as pid 23849
I20260620 23:12:39.220790 1153 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTMrFo/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996897900784-1153-0/minicluster-data/master-2/wal/instance
I20260620 23:12:39.220960 1153 external_mini_cluster.cc:1428] Setting key 3e2e47374b9a49ac4506774979746fea
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:12:39.229600 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.221221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.1.32.126:34863)
0620 23:12:39.221282 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.221284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.221524 (+ 240us) server_negotiation.cc:407] Connection header received
0620 23:12:39.222000 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.222002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.222014 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.222078 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.222646 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.223174 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.224041 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.224222 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.224324 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.226552 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.226553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.226554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.226561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.228252 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.228664 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.228665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.228665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.228720 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.228971 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.228972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.228972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.229089 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.229156 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.229434 (+ 278us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.229514 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:12:39.229909 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.221213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.126:34863 (local address 127.0.0.1:34686)
0620 23:12:39.221371 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.221416 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.221885 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.222115 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.222117 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.222408 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.222552 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.222556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.223279 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.223280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.223941 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.223943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.224355 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.224357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.224466 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.224965 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.224975 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.226464 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.228346 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.228352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.228354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.228590 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.228792 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.228793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.228794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.228900 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.229162 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.229169 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.229314 (+ 145us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.229401 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.229542 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260620 23:12:39.229918 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.221314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55998 (local address 127.1.32.124:34269)
0620 23:12:39.221419 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.221427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.221567 (+ 140us) server_negotiation.cc:407] Connection header received
0620 23:12:39.221900 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.221910 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.221930 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.222020 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.222675 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.223499 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.224226 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.224524 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.224602 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.227059 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.227066 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.227066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.227073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.228533 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.228875 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.228877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.228878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.228936 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.229127 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.229128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.229128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.229315 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.229465 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.229549 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.229620 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":80,"thread_start_us":51,"threads_started":1}
I20260620 23:12:39.230692 23917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.126@KRBTEST.COM
I20260620 23:12:39.236788 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.230927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.125:41775 (local address 127.0.0.1:50226)
0620 23:12:39.231030 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.231040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.231084 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.231185 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.231185 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.231325 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.231363 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.231365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.231893 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.231894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.232332 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.232333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.232682 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.232683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.232738 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.233094 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.233099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.234180 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.235957 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.235958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.235958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.236115 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.236314 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.236315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.236315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.236360 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.236613 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.236613 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.236641 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.236682 (+ 41us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.236737 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260620 23:12:39.236811 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.231013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.1.32.125:41775)
0620 23:12:39.231065 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.231067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.231075 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:39.231135 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.231136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.231143 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.231179 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.231425 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.231820 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.232418 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.232602 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.232683 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.234279 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.234280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.234280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.234288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.235881 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.236186 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.236187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.236187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.236241 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.236417 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.236417 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.236418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.236553 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.236611 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.236708 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.236757 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:39.237160 23917 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:12:39.237937 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.230764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.1.32.126:34863)
0620 23:12:39.230850 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.230852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.230879 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:12:39.230957 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.230958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.230967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.231021 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.231588 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.231982 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.232579 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.232757 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.232820 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.235098 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.235099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.235099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.235105 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.236553 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.237161 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.237162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.237163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.237211 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.237411 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.237411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.237412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.237546 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.237707 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.237773 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.237857 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.124@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.32.125@KRBTEST.COM
I20260620 23:12:39.243579 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.237401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:56010)
0620 23:12:39.237436 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.237445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.237474 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.237605 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.237606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.237747 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.237778 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.237779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.238578 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.238579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.239194 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.239196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.239543 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.239544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.239591 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.239947 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.239951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.240968 (+ 1017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.242495 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.242496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.242496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.242693 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.242956 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.242957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.242957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.243019 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.243316 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.243316 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.243350 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.243403 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.243464 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 23:12:39.243692 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.237429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56010 (local address 127.1.32.124:34269)
0620 23:12:39.237467 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.237469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.237476 (+ 7us) server_negotiation.cc:407] Connection header received
0620 23:12:39.237515 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.237516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.237542 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.237601 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.237850 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.238520 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.239267 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.239464 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.239528 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.241065 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.241066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.241066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.241073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.242412 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.242770 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.242771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.242772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.242827 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.243102 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.243103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.243104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.243244 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.243317 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.243452 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.243528 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 23:12:39.245476 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.238516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.1.32.125:41775)
0620 23:12:39.238546 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.238548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.238556 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:39.238597 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.238598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.238606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.238664 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.239114 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.239731 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.240419 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.240616 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.240695 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.242701 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.242702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.242702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.242708 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.244449 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.244797 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.244798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.244798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.244845 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.245016 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.245017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.245017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.245103 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.245204 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.245333 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.245385 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 23:12:39.245491 23917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: Bootstrap starting.
I20260620 23:12:39.246544 23917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:39.246884 23917 log.cc:824] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:39.246979 1153 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:12:39.247690 23917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: No bootstrap required, opened a new log
I20260620 23:12:39.247916 23741 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fe5bb98165024583a3486646a1dadf35"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34700
I20260620 23:12:39.249145 23917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.249270 23917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36cb4182a4cb476882ab14fa3dc816c3, State: Initialized, Role: FOLLOWER
I20260620 23:12:39.249353 23917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.249634 23917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:39.249642 23923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.249759 23923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 23:12:39.251439 23934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:39.251494 23934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:39.251704 23849 master_runner.cc:186] Error getting master registration for 127.1.32.126:34863: OK, instance_id { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" instance_seqno: 1781997158919381 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:12:39.253931 23849 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:12:39.262727 23767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } attempt: 2
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997158, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.126@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:12:39.270681 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.263031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:56012)
0620 23:12:39.263103 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.263115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.263158 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.263321 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.263321 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.263504 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.263555 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.263556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.264501 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.264502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.265320 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.265322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.265791 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.265792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.265849 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.266332 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.266337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.267762 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.269566 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.269567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.269568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.269778 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.269988 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.269989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.269989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.270048 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.270366 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.270367 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.270406 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.270480 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.270585 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 23:12:39.270702 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.263091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.1.32.124:34269)
0620 23:12:39.263146 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.263148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.263157 (+ 9us) server_negotiation.cc:407] Connection header received
0620 23:12:39.263234 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.263235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.263245 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.263309 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.263638 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.264400 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.265411 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.265714 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.265785 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.267845 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.267846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.267847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.267853 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.269482 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.269865 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.269866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.269866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.269924 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.270113 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.270114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.270114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.270282 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.270363 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.270536 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.270608 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:39.272228 23767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35: Bootstrap starting.
I20260620 23:12:39.272892 23767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:39.273187 23767 log.cc:824] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:39.274008 23767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35: No bootstrap required, opened a new log
I20260620 23:12:39.275393 23767 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.275527 23767 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe5bb98165024583a3486646a1dadf35, State: Initialized, Role: FOLLOWER
I20260620 23:12:39.275607 23767 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.275918 23767 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:39.275889 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.275983 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:39.275933 23741 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 23:12:39.276049 23741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:39.276758 23741 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.276979 23741 leader_election.cc:288] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [CANDIDATE]: Term 1 election: Requested vote from peers e784f45688ac41f09402180ca3049b3f (127.1.32.125:41775), 36cb4182a4cb476882ab14fa3dc816c3 (127.1.32.124:34269)
I20260620 23:12:39.277308 23891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe5bb98165024583a3486646a1dadf35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "36cb4182a4cb476882ab14fa3dc816c3"
I20260620 23:12:39.277413 23891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:39.278266 23891 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5bb98165024583a3486646a1dadf35 in term 1.
I20260620 23:12:39.278481 23716 leader_election.cc:302] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cb4182a4cb476882ab14fa3dc816c3, fe5bb98165024583a3486646a1dadf35; no voters:
I20260620 23:12:39.278604 23937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:12:39.278743 23937 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [term 1 LEADER]: Becoming Leader. State: Replica: fe5bb98165024583a3486646a1dadf35, State: Running, Role: LEADER
I20260620 23:12:39.278843 23937 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.279238 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe5bb98165024583a3486646a1dadf35. Latest consensus state: current_term: 1 leader_uuid: "fe5bb98165024583a3486646a1dadf35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.279338 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:39.280400 23958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:12:39.279377 23699 master_runner.cc:186] Error getting master registration for 127.1.32.125:41775: OK, instance_id { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" instance_seqno: 1781997159059535 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:12:39.282014 23699 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:12:39.283118 23958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:12:39.287204 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.278254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.1.32.126:34863)
0620 23:12:39.278291 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.278293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.279389 (+ 1096us) server_negotiation.cc:407] Connection header received
0620 23:12:39.279532 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.279533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.279543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.279575 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.280345 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.281044 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.282152 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.282456 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.282492 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.284229 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.284230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.284231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.284238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.285894 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.286274 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.286276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.286276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.286327 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.286547 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.286547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.286548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.286699 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.286764 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.287004 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.287077 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:12:39.290320 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.279255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.1.32.124:34269)
0620 23:12:39.282084 (+ 2829us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.282086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.282203 (+ 117us) server_negotiation.cc:407] Connection header received
0620 23:12:39.282290 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.282291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.282300 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.282339 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.283083 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.283810 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.284800 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.285079 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.285115 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.286480 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.286481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.286481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.286487 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.288569 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.289244 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.289245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.289246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.289304 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.289758 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.289760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.289760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.289883 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.289947 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.290147 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.290232 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2810}
I20260620 23:12:39.291250 23891 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [term 1 FOLLOWER]: Refusing update from remote peer fe5bb98165024583a3486646a1dadf35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:39.291731 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.278776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.1.32.125:41775)
0620 23:12:39.278841 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.278843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.279622 (+ 779us) server_negotiation.cc:407] Connection header received
0620 23:12:39.279749 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.279750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.279773 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.279804 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.281836 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.282573 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.286010 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.286824 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.286861 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.288702 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.288703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.288704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.288711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.290300 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.290949 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.290950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.290950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.291010 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.291209 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.291210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.291211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.291331 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.291399 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.291578 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.291657 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 23:12:39.292040 23954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:39.293243 23923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe5bb98165024583a3486646a1dadf35. Latest consensus state: current_term: 1 leader_uuid: "fe5bb98165024583a3486646a1dadf35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.293324 23923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:39.295779 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe5bb98165024583a3486646a1dadf35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.295822 23842 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } attempt: 3
I20260620 23:12:39.295878 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:39.295948 23923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe5bb98165024583a3486646a1dadf35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.296007 23923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:39.296132 23961 mvcc.cc:205] Tried to move back new op lower bound from 7299060364454715392 to 7299060364406902784. Current Snapshot: MvccSnapshot[applied={T|T < 7299060364454715392}]
Jun 20 23:12:39 dist-test-slave-fzjm krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781997159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.32.125@KRBTEST.COM for kudu/127.1.32.124@KRBTEST.COM
I20260620 23:12:39.303983 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.296015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.32.124:34269 (local address 127.0.0.1:56030)
0620 23:12:39.296089 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 23:12:39.296098 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 23:12:39.296144 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:12:39.296476 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:12:39.296477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:12:39.296616 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:12:39.296652 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.296654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.297551 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.297552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.298357 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:12:39.298359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.298861 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.298862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:12:39.298929 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.299618 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:12:39.299622 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:12:39.301039 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:12:39.302774 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.302775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.302775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.303009 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.303305 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:12:39.303305 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:12:39.303306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:12:39.303370 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:12:39.303709 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:12:39.303710 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:12:39.303760 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 23:12:39.303814 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 23:12:39.303895 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 23:12:39.304751 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:39.296112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56030 (local address 127.1.32.124:34269)
0620 23:12:39.296345 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.296347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.296353 (+ 6us) server_negotiation.cc:407] Connection header received
0620 23:12:39.296380 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.296380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.296388 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.296441 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.296753 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.297466 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.298483 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:39.298765 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:39.298814 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:39.301130 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:12:39.301131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:12:39.301131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:12:39.301136 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:12:39.302675 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.303091 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.303092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.303093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.303199 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:12:39.303440 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:12:39.303440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:12:39.303441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:12:39.303622 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:12:39.304426 (+ 804us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:39.304506 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 23:12:39.304623 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260620 23:12:39.306267 23842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: Bootstrap starting.
I20260620 23:12:39.306924 23842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: Neither blocks nor log segments found. Creating new log.
I20260620 23:12:39.307226 23842 log.cc:824] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:12:39.308198 23842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: No bootstrap required, opened a new log
I20260620 23:12:39.309990 23842 raft_consensus.cc:348] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.310112 23842 raft_consensus.cc:729] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e784f45688ac41f09402180ca3049b3f, State: Initialized, Role: FOLLOWER
I20260620 23:12:39.310180 23842 consensus_queue.cc:260] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } }
I20260620 23:12:39.310515 23816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fe5bb98165024583a3486646a1dadf35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e784f45688ac41f09402180ca3049b3f"
I20260620 23:12:39.310513 23815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:12:39.311440 23815 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [term 1 FOLLOWER]: Refusing update from remote peer fe5bb98165024583a3486646a1dadf35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:12:39.311728 23937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:12:39.312244 23816 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe5bb98165024583a3486646a1dadf35 in term 1.
I20260620 23:12:39.310457 23842 sys_catalog.cc:564] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:12:39.313431 23972 sys_catalog.cc:455] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe5bb98165024583a3486646a1dadf35. Latest consensus state: current_term: 1 leader_uuid: "fe5bb98165024583a3486646a1dadf35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.313537 23972 sys_catalog.cc:458] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:39.313589 23972 sys_catalog.cc:455] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe5bb98165024583a3486646a1dadf35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.313633 23972 sys_catalog.cc:458] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:39.314224 23958 catalog_manager.cc:1347] Generated new cluster ID: 8dccfd7a81154640b26f97411fa5d98c
I20260620 23:12:39.314276 23958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260620 23:12:39.314812 23981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:12:39.314850 23981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:12:39.310442 23966 sys_catalog.cc:455] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.316512 23966 sys_catalog.cc:458] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:12:39.317027 23968 mvcc.cc:205] Tried to move back new op lower bound from 7299060364454715392 to 7299060364406902784. Current Snapshot: MvccSnapshot[applied={T|T < 7299060364454715392}]
I20260620 23:12:39.326511 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe5bb98165024583a3486646a1dadf35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe5bb98165024583a3486646a1dadf35" member_type: VOTER last_known_addr { host: "127.1.32.126" port: 34863 } } peers { permanent_uuid: "e784f45688ac41f09402180ca3049b3f" member_type: VOTER last_known_addr { host: "127.1.32.125" port: 41775 } } peers { permanent_uuid: "36cb4182a4cb476882ab14fa3dc816c3" member_type: VOTER last_known_addr { host: "127.1.32.124" port: 34269 } } }
I20260620 23:12:39.326606 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35 [sys.catalog]: This master's current role is: LEADER
I20260620 23:12:39.334743 23958 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:12:39.335476 23958 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:12:39.342692 23958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe5bb98165024583a3486646a1dadf35: Generated new TSK 0
I20260620 23:12:39.342864 23958 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:12:39.375105 23770 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.373275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.1.32.126:34863)
0620 23:12:39.373319 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.373322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.374321 (+ 999us) server_negotiation.cc:407] Connection header received
0620 23:12:39.374532 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.374533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.374546 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.374577 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.375037 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34718: BlockingRecv error: recv got EOF from 127.0.0.1:34718 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 23:12:39.375941 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.373689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.1.32.125:41775)
0620 23:12:39.373728 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.373730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.374477 (+ 747us) server_negotiation.cc:407] Connection header received
0620 23:12:39.375495 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.375497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.375507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.375536 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.375883 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50268: BlockingRecv error: recv got EOF from 127.0.0.1:50268 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 23:12:39.376896 23921 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.373991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.1.32.124:34269)
0620 23:12:39.374028 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.374030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.376364 (+ 2334us) server_negotiation.cc:407] Connection header received
0620 23:12:39.376480 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.376481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.376512 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.376542 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.376837 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56038: BlockingRecv error: recv got EOF from 127.0.0.1:56038 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 23:12:39.379295 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.378568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50274 (local address 127.1.32.125:41775)
0620 23:12:39.378601 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.378602 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.378786 (+ 184us) server_negotiation.cc:407] Connection header received
0620 23:12:39.378906 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.378907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.378925 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.378954 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.379254 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50274: BlockingRecv error: recv got EOF from 127.0.0.1:50274 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:12:39.382439 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.381703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.1.32.125:41775)
0620 23:12:39.381734 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.381736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.381951 (+ 215us) server_negotiation.cc:407] Connection header received
0620 23:12:39.382043 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.382043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.382053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.382081 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.382399 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50284: BlockingRecv error: recv got EOF from 127.0.0.1:50284 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:12:39.386276 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.385617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.1.32.125:41775)
0620 23:12:39.385648 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.385649 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.385832 (+ 183us) server_negotiation.cc:407] Connection header received
0620 23:12:39.385916 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.385917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.385926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.385952 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.386237 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50292: BlockingRecv error: recv got EOF from 127.0.0.1:50292 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:12:39.392107 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.391429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50294 (local address 127.1.32.125:41775)
0620 23:12:39.391458 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.391460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.391670 (+ 210us) server_negotiation.cc:407] Connection header received
0620 23:12:39.391760 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.391761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.391770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.391796 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.392069 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50294: BlockingRecv error: recv got EOF from 127.0.0.1:50294 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 23:12:39.402025 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.401270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50298 (local address 127.1.32.125:41775)
0620 23:12:39.401302 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.401304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.401525 (+ 221us) server_negotiation.cc:407] Connection header received
0620 23:12:39.401617 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.401618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.401628 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.401655 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.401983 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50298: BlockingRecv error: recv got EOF from 127.0.0.1:50298 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 23:12:39.420164 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.419474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50310 (local address 127.1.32.125:41775)
0620 23:12:39.419516 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.419518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.419656 (+ 138us) server_negotiation.cc:407] Connection header received
0620 23:12:39.419691 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.419692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.419703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.419770 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.420099 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50310: BlockingRecv error: recv got EOF from 127.0.0.1:50310 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 23:12:39.454535 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.453903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.1.32.125:41775)
0620 23:12:39.453948 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.453950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.453958 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:39.453984 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.453985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.453995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.454050 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.454470 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50318: BlockingRecv error: recv got EOF from 127.0.0.1:50318 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 23:12:39.520632 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.519914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.1.32.125:41775)
0620 23:12:39.519985 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.519987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.520111 (+ 124us) server_negotiation.cc:407] Connection header received
0620 23:12:39.520157 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.520158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.520168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.520219 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.520549 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50322: BlockingRecv error: recv got EOF from 127.0.0.1:50322 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 23:12:39.650518 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.649798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.1.32.125:41775)
0620 23:12:39.649857 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.649859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.649942 (+ 83us) server_negotiation.cc:407] Connection header received
0620 23:12:39.649989 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.649990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.649999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.650058 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.650410 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38612: BlockingRecv error: recv got EOF from 127.0.0.1:38612 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260620 23:12:39.908470 23847 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:12:39.907637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.1.32.125:41775)
0620 23:12:39.907714 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:39.907716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:39.907799 (+ 83us) server_negotiation.cc:407] Connection header received
0620 23:12:39.907935 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:39.907936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:39.907946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:39.908012 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:12:39.908400 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38620: BlockingRecv error: recv got EOF from 127.0.0.1:38620 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:12:40.251922 23934 catalog_manager.cc:1259] Loaded cluster ID: 8dccfd7a81154640b26f97411fa5d98c
I20260620 23:12:40.251996 23934 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: loading cluster ID for follower catalog manager: success
I20260620 23:12:40.252740 23934 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: acquiring CA information for follower catalog manager: success
I20260620 23:12:40.252978 23934 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36cb4182a4cb476882ab14fa3dc816c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:12:40.315258 23981 catalog_manager.cc:1259] Loaded cluster ID: 8dccfd7a81154640b26f97411fa5d98c
I20260620 23:12:40.315495 23981 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: loading cluster ID for follower catalog manager: success
I20260620 23:12:40.316309 23981 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: acquiring CA information for follower catalog manager: success
I20260620 23:12:40.316675 23981 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e784f45688ac41f09402180ca3049b3f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:12:40.424019 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:40.421576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.1.32.125:41775)
0620 23:12:40.421724 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:40.421726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:40.421737 (+ 11us) server_negotiation.cc:407] Connection header received
0620 23:12:40.421803 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:40.421804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:40.421855 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:40.421941 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:40.422073 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.422565 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.423332 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.423526 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.423587 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:40.423745 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:40.423848 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:40.423903 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:40.423935 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 23:12:40.423953 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260620 23:12:40.427906 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:40.425791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.1.32.124:34269)
0620 23:12:40.425878 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:40.425879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:40.425887 (+ 8us) server_negotiation.cc:407] Connection header received
0620 23:12:40.425940 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:40.425941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:40.425974 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:40.426041 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:40.426121 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.426567 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.427252 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.427436 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.427495 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:40.427603 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:40.427696 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:40.427748 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:40.427776 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 23:12:40.427791 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":44,"threads_started":1}
I20260620 23:12:40.432200 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:40.429505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.1.32.125:41775)
0620 23:12:40.429545 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:40.429546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:40.429662 (+ 116us) server_negotiation.cc:407] Connection header received
0620 23:12:40.429757 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:40.429758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:40.429768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:40.429821 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:40.429964 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.430540 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.431479 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.431746 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.431803 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:40.431948 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:40.431976 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:40.432024 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:40.432077 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 23:12:40.432095 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 23:12:40.432430 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:40.429733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.1.32.124:34269)
0620 23:12:40.429766 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:40.429767 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:40.430021 (+ 254us) server_negotiation.cc:407] Connection header received
0620 23:12:40.430103 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:40.430104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:40.430114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:40.430158 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:40.430272 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.430953 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.431738 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.431985 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.432034 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:40.432187 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:40.432214 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:40.432238 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:40.432332 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 23:12:40.432351 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 23:12:40.432548 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:12:40.429495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.1.32.126:34863)
0620 23:12:40.429591 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 23:12:40.429593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 23:12:40.429703 (+ 110us) server_negotiation.cc:407] Connection header received
0620 23:12:40.429750 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:12:40.429751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:12:40.429802 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:12:40.429877 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:12:40.430029 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.430802 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.431683 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:12:40.431950 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:12:40.432004 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:12:40.432219 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:12:40.432320 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:12:40.432353 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0620 23:12:40.432447 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 23:12:40.432467 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":52,"threads_started":1}
I20260620 23:12:40.438475 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23699
I20260620 23:12:40.443576 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23774
I20260620 23:12:40.448136 1153 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTMrFo/build/release/bin/kudu with pid 23849
2026-06-20T23:12:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2761 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5553 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (262552 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST